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ABSTRACT 

Factors contributing to program success or failure were 
evaluated at 16 English colleges. Questionnaires were distributed to 231 
teachers with 59 of those questioned further during an interview. Respondents 
emphasized the importance of good teaching, committed and knowledgeable 
teachers, and effective and appropriate processes for recruitment, 
orientation, assessment, tutoring, and student support. They agreed on 
factors, typically outside teacher control, that can have a negative impact 
on student achievement, such as college resources, changes to programs of 
study, and course scheduling. Follow-up interviews revealed that teachers in 
programs with high student retention and achievement rates based their 
teaching on interaction and partnership with students and believed that 
effective teachers adapt and develop their teaching to meet student needs. 
Equally significant was the finding that teachers in successful programs 
worked co.llaboratively with their peers as well as with college 
administration to achieve their goals. Teachers in programs with low student 
success rates blamed their supervisors, students, or both. They suggested 
that the students, not the teaching methodology, needed to change. 

Suggestions for improving unsuccessful programs included: (1) engaging 

teacher's value systems and their concept of teaching role; (2) working with 
teachers collaboratively ; and, (3) reexamining the role of college 
administration as it affects teachers. Appended are a list of colleges that 
participated; a description of research methods; letter to survey 
respondents; questionnaire; guidance notes for college coordinators; and a 
data sheet for each course/program. (Contains 14 tables and 17 references.) 
(AJ) . . 
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Summary of key messages 



• There are strong elements of similarity and continuity in the views of college 
teachers drawn from a wide variety of relatively successful and unsuccessful 
courses and programmes. 

• Teachers emphasise the importance of good teaching, committed and 
knowledgeable teachers and effective and appropriate processes for 
recruitment, induction, assessment, tutoring, quality assurance and student 
support. 

• Teachers also agree the factors with a negative impact on student outcomes. 
These are factors, which are typically outside teachers’ influence or control: 
resources, changes to the student cohort, timetabling and course administration. 

• There are also some evident discontinuities. Teachers from courses with the 
highest and improving student success rates were most optimistic about the 
impact of changes. 

• By contrast, teachers from courses where once high student success rates were 
in decline were the most pessimistic about change. 

• In the survey, and contrary to expectations, the views of teachers from low 
baseline, declining courses tended to resemble the views expressed by teachers 
from high baseline, improving courses. 

• Contrary to expectations, teachers from low baseline, improving courses were 
not particularly optimistic about the impact of changes. 

• The follow-up interviews helped to explain these apparently paradoxical findings. 

• Teachers on courses with high student retention and achievement rates seem to 
base their teaching on interaction and partnership with their students. In their 
view, effective teachers adapt and develop their teaching to meet the needs of 
their learners. 

• Teachers from courses with low student success rates which are not improving 
or which are declining, seem to blame either their managers, their students or 
both. They suggested it is not their teaching that needs to change, but rather 
their students. 

• The interviews also point to the importance of unwritten or implied contracts in 
colleges. Teachers from the most successful courses emphasise in particular 
the contracts between members of teaching teams and between teachers and 
their students. 

• Teachers from the least successful courses tend to stress the importance of 
implicit contracts with college managers, which have broken down. 

• Teaching teams appear to have an ambivalent role. Effective, confident and 
autonomous teams appear to sustain and enthuse teachers from the most 
successful courses. In unsuccessful courses, teams seem to support and 
confirm the view that there is little or no scope for improvement. 

• Efforts to improve colleges that do not engage teachers’ value systems and the 
ways that they conceive of the teaching role, are not likely to be successful. 

• Improvement strategies that seek to engage teachers will also need to engage 
the teams within which teaching is planned and delivered. 



College managers need to renew their unwritten or implied contracts with 
teachers if they want teachers, in turn, to renew and develop their partnership 
with students. 



Introduction to the research 



The issue of how to improve the quality of colleges currently has high political, 
professional and managerial priority. Although colleges remain under-researched, 
compared with schools, there is a large and growing literature in the public domain and 
in the grey area of unpublished research dissertations and research produced for 
internal consumption with colleges (Martinez 2001). The research reported here was 
designed to shed some new light on college improvement issues by focusing on what 
could be seen as the engine room of improvement: individual courses and 
programmes. 

The research question can be put quite simply: 

• What makes the most difference between courses or programmes where learner 
performance improves and courses and programmes where learner 
performance declines? 

Subsidiary questions included the following: 

• Are there differences between courses where learner performance improves 
from a high baseline position, compared to courses which improve from a low 
baseline position? 

• Are there differences between courses where learner performance declines from 
a high baseline position compared with courses where learner performance 
declines from a low baseline position? 

• Where learner performance improves, do teachers tend to attribute the 
improvement to their own agency and discount the importance of factors that lie 
outside their control? 

• Where learner performance is declining, do teachers tend to attribute the 
deterioration to factors outside their control and discount the importance of their 
own agency? 

Research design 

The research was undertaken in two stages: quantitative and qualitative. In the 
quantitative stage, invitations were sent to around 40 colleges, which had participated 
in at least two of the first three rounds of action research projects to improve retention 
and achievement. These projects were supported by the Raising Quality and 
Achievement Programme, founded by DfES and led by LSDA (for discussions of this 
programme, see Martinez 2000a and Cousin 2002). 

The invitations were thus addressed to colleges that were known to be actively 
interested in improvement issues. A group of 16 colleges was established, comprising 
13 FE and tertiary colleges and three sixth form colleges. The colleges are 
geographically spread across England and included colleges that operate in a range of 
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local contexts from inner city to small town and rural locations. The colleges that took 
part in the research are listed in Appendix 1 . 

College coordinators attended two seminars to determine the: 

• overall research design 

• design of the questionnaire 

• criteria for the selection of the sample 

• process of administering the questionnaire. 

• A more detailed note of the research design and process is attached as 
Appendix 2. The survey instruments are also included in appendices as follows: 

• questionnaire: Appendix 3 

• covering letter to accompany the questionnaire: Appendix 4 

• guidance notes for college coordinators: Appendix 5 

• example of data sheet for each course/programme: Appendix 6. 

College coordinators selected teachers who would be invited to complete the 
questionnaire. Teachers were selected from a cross-section of courses with high or low 
baseline positions. In this context, baseline position means high or low success rates in 
the academic year 1996/7 were relative to the norm for each college. Success rates 
were chosen as the best single indicator of the relative success of a course. The 
success rate is the proportion of students who start (from the first census date of the 
first year of a course) and go on to achieve their qualification. In other words, success 
rates can be calculated by multiplying retention rates by achievement rates. Two 
further selection criteria were that teachers had taught on the course for at least 2 
years and that the course or programme should have a minimum group size of 12 
starters (to minimise the effect of random variations). 

College coordinators distributed the questionnaire and returned the completed 
questionnaire to LSDA, together with data sheets giving performance indicator 
information for each of the courses or programmes included in the survey. The analysis 
of the survey data was provided to college coordinators in the summer of 2001. The 
analysis was not disseminated, to avoid prejudicing the follow-up interviews that were 
arranged by the coordinators and took place between September and December 2001. 
Interviews were completed with 59 teachers, who were available on the day of the visit 
to the college and who were willing to be interviewed. Almost all the teachers were 
interviewed separately. There was no evident bias towards more or less successful 
courses in the interviews: the interview sample resembled the questionnaire sample. 
The interviews covered the same topics as the survey, but were otherwise 
unstructured. Interviewees were assured that their views would be treated in 
confidence and that no individual teacher, course or programme would be identified in 
any publication arising from the research. Judging by the forthright way in which 
teachers expressed their views, they appear to have had confidence in these 
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arrangements and not to have felt constrained by the interview process. These 
interviews comprised the qualitative phase of the research. 

Structure of the report 

The structure of the report follows the two distinct phases of the research. The first part 
introduces and analyses the outcomes of the quantitative, survey phase. It draws some 
conclusions and poses some questions, which are then addressed in the second part, 
which examines and reports on the interviews with college teachers. 
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PART 1 QUANTITATIVE RESEARCH 
Introduction 



There are already many case studies of college improvement. Over 220 improvement 
projects generated by the Raising Quality and Achievement (RQA) Programme are 
available at www.rqa.org.uk. There are also syntheses of college improvement 
strategies based on case studies generated by the RQA Programme (Martinez 2000b 
and 2001, Cousin 2002). There are also some syntheses of improvement activities that 
have occurred outside the RQA Programme (Kenwright 1997, Martinez 1996 and 1997, 
National Audit Office 2001 ). 

The case studies have the weaknesses and strengths associated with an action 
research approach. They are empirically based, collaborative, real world projects. They 
have a strong focus on improvement and the transfer of successful practice, together 
with a level of detail sufficient to facilitate replication and transfer and to allow 
practitioners to make their own judgements about relevance. On the other hand, they 
are also variable in their method, the rigour of their analysis and sometimes the 
robustness of their data. It is difficult, moreover, to derive generalisations concerning 
‘good’ (still less ‘best’) practice, transferability to different context and, sometimes, 
cause and effect relationships. The case studies do not distinguish easily those 
strategies that have had a particularly large impact on retention/achievement and those 
with a relatively small impact. Finally, because of an understandable reluctance to 
report improvement efforts that have failed, the improvement case studies lack a 
control group of colleges where improvement efforts have not been successful. 

The aim, therefore, of this part of the report is to extend the field of college 
improvement research by applying more quantitative research techniques to a 
structured sample of courses within colleges. 

The research design 

The research design was inspired by some research undertaken by John Gray and 
others which is published as Improving schools (Gray et al. 1999). The methodological 
innovation developed by Gray combined elements of methodology from both school 
effectiveness and school improvement traditions. A sample of schools was selected to 
include schools with different baseline positions and different rates of improvement. A 
cross-section of staff in each of the sample schools was surveyed to gather information 
about different causative factors and process variables. 

The relationships between the school type (baseline position and degree of 
improvement) and the causative factors and processes were explored through 
statistical analysis. Gray concluded by drawing three tentative conclusions: 
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. . . ‘rapid’ improvement was linked to changes in the schools’ use 
of various tactics for maximising exam grades, the development 
of policies for supporting teaching and learning; the degree to 
which they have begun to tackle the process of teaching and 
learning at classroom level; and the extent to which 
responsibilities have been given to pupils 

schools which had improved more ‘rapidly’ had higher proportions 
of teachers reporting that there had been ‘substantial’ changes in 
the quality of teaching and learning over the last 5 years. In 
‘slowly’ improving schools only a very small proportion of the staff 
(typically around 1 in 10 or fewer) reported similar changes 

there are some suggestions that while schools seem to have 
launched change initiatives on many fronts, their approaches to 
improvement could be bundled together. Schools which did some 
things also did others. The most rapidly improving schools seem, 
however, to have found ways of straddling distinctively different 
approaches at the same time 

Gray et al. 1999 pi 35 



Research hypotheses 

Earlier work on teachers’ attitudes in further education (Martinez 1999) found that: 

• factors to do with resources, the nature of the student cohort and college 
management are generally perceived to have a negative impact on 
student achievement 

• factors to do with teachers and teaching are generally perceived to have 
a positive impact on student achievement 

• teachers who made more pessimistic observations about their work 
tended to emphasise the influence of negative factors on student 
achievement; teachers who tended to make more optimistic judgements 
tended to emphasise the importance of teachers and teaching on student 
achievement 

• the issue is by no means clear-cut: teacher attitudes are highly individual 
and colleagues in the same teaching team may have diametrically 
opposed views. 

Research on staff opinions conducted in the context of work on student retention and 
achievement came to broadly similar conclusions. In the main, teachers on 
programmes with relatively poor student outcomes (in terms of retention and 
achievement) tended to explain these in terms of external factors. At the other extreme, 
teachers tend to attribute high retention and achievement rates to their own agency 
(Martinez and Munday 1998 pp75-6). 
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In the light of these findings and Gray’s work, the following hypotheses were proposed: 



• there would be different patterns of response between teachers working 
on different categories of course 

• the most marked differences would occur between courses where results 
were improving and those where results were declining. 

• teachers on improving courses would tend to attribute improvements to 
their own agency and to factors over which they have most control 

• teachers on courses where results were declining would tend to attribute 
the decline to factors over which they had little or no control 

• the baseline position of courses would affect the patterns of response, but 
there were no specific expectations concerning such patterns. 

Analysis of the response 



We received 231 completed questionnaires for 136 different courses/programmes. In 
terms of level of programme, these split into the following proportions (rounded to 
whole figures): 

• Level 1 (14%) 

• Level 2 (25%) 

• Level 3 (62%) 

Four types of qualification accounted for three-quarters of the completed 
questionnaires: 



• GNVQ (27%) 

• NVQ (14%) 

• AS/A-level (22%) 

• National Diploma (12%) 



The questionnaires were mainly completed by teachers who were organised in 
relatively small teams and teaching on full-time courses undertaken by 16-19-year-old 
students: 



• 83% of the courses were full-time, 1 7% part-time 

• the range of hours of teaching per week per course was from 2 to 
30; 16 hours was the mode and 14.25 hours was the median 

• roughly half the courses lasted for 1 year and half for 2 years 

• 80% of courses were primarily for 16-19 year olds and 20% for 
students aged 1 9 plus 

• 5% of courses had one teacher; 32% had 2-3 teachers; 36% had 4- 
5 teachers; 25% had 6-10 teachers and 2% had more than 10 
teachers. 
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Many of the respondents had two or more roles. They described their roles in the 
following ways: teacher (66%), tutor (57%), course leader (45%), other (8%). The roles 
add up to more than 100% since respondents could select any and all of the roles. 

A further analysis was undertaken to investigate whether respondents who were 
course team leaders expressed different views from teachers, tutors and other 
respondents. In general terms, teachers and tutors tended to give slightly more positive 
assessments of the impact of change than course team leaders, although this was not 
usually statistically significant, even at the 10% level. 

The exception to this observation occurred in respect of changes to the student 
cohorts. Here, course team leaders were significantly less negative in their assessment 
of the impact of changes to the student cohort, in terms of students’ motivation, ability, 
learning skills, clarity of career goals, their personal problems and how near they lived 
to college. With this notable exception, it seemed safe to conclude that course team 
leaders have broadly similar views to those of the teachers, tutors and other members 
of staff included in the quantitative research. 



Overall findings 

Year-on-year stability 

One major finding is that success rates on the different courses and programmes 
varied considerably over the 3 years of the survey, and more than could be accounted 
for by changes to teaching, teachers or student cohort or other factors. This implies a 
substantial amount of more-or-less random variation at course or programme level. 
There are two further implications. First, the degree of year-on-year variation tends to 
‘muddy’ the data and imposes a requirement for a cautious and careful interpretation. 
Second, and in the context of management concerns to improve performance, 
too much significance should not be attached to a single and perhaps untypical 
year’s retention and achievement rates for an individual course. 

Success rates varied considerably from year to year. On the assumption that ‘stability’ 
implies an annual upward or downward movement of no more than 10%, performance 
at course level seems to be quite unstable. Specifically: 

• only a quarter of courses had stable success rates over all three 
years 

• just over half (53%) had success rates which were stable over two 
years 

• just over a fifth (21%) of courses had success rates which varied 
by more than 10% in two successive years 

• consistent success rates over three years could be at high or low 
levels 
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• where success rates were stable for two years, the ‘rogue’ year 
could be in any one of the three years and the ‘rogue’ success rate 
could be above or below those of the other two years 

• where success rates were unstable, this could be with an upward 
or downward trend, or with no discernible trend either way. 

The extent of year-on-year variation prompts the question: does the presence or lack of 
stability colour teachers’ views? To investigate this question, two further analyses were 
undertaken with a specific focus on those high and low baseline courses that had 
maintained their performance over the 3 years. The hypothesis to be tested was 
whether greater stability was associated with a more positive assessment of the impact 
of various change factors. 

This proved not to be the case. There were relatively few occasions where the views of 
teachers from the more stable courses proved to be different from the others in a 
statistically significant way (even at the 10% level). Even where differences could be 
detected, there was no pronounced pattern: teachers on the more stable courses were 
more optimistic about some and less optimistic about other change factors, compared 
with teachers on courses with less consistent outcomes. 

Rates of change over the 3-year period 

Before asking teachers to assess the positive or negative impact of changes, they were 
asked to score each factor for the rate of change. The choice that was offered was 
between little or no change and significant change. Somewhat to our surprise, most 
respondents scored most factors as having changed only a little or not at all. 

Factors that a third or more of respondents identified as changing significantly are listed 
below. The figures in brackets refer to the percentage of respondents who agreed that 
the factor had changed significantly. As we shall see, most of the changes seen as 
significant are also identified with a negative impact on retention and achievement. 
Only three changes are generally seen as having a positive: emphasis on improving 
teaching and learning, observation of teaching and learning and amount of support for 
at-risk students. The changes that most respondents agreed had changed significantly 
included: amount of course administration, time for preparation, morale and number of 
teaching hours. All of these are seen as having an adverse effect on student retention 
and achievement: 



• amount of course administration (54) 

• time you have available for preparation (52) 

• morale within your team (48) 

• morale within your department or faculty (48) 

• membership of your course/programme team (42) 

• number of teaching hours you have (40) 

• emphasis placed on improving teaching and learning (38) 

• observation of teaching and learning (37) 
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• ability of students (36) 

• amount of paid part-time work done by students (36) 

• arrangements for key skills (33) 

• your departmental/faculty headship (33) 

• administrative/clerical support (33) 

• time available for team meetings (33). 

In response to the final, summative question, respondents were asked to identify the 
changes having the largest negative or positive impact. Again, figures in brackets refer 
to the percentage of respondents who identified the change as having a particularly 
negative impact: 

• resources (53%) 

• support for teachers (40%) 

• student cohort (34%) 

• timetabling (31%) 

• course organisation and administration (25%). 

The changes with the most positive impact were identified as follows: 

• support for students (45%) 

• teaching (44%) 

• personal tutoring (38%) 

• staffing (38%) 

• assessment and feedback (31%). 

Two alternative but compatible inferences can be made concerning teachers’ general 
perceptions of change. Major changes tend to be associated with negative impacts on 
student outcomes. The factors that were most widely seen to be changing significantly 
are mainly seen as having a negative impact on retention and achievement. These 
findings provide some support to calls to reduce bureaucracy and provide 
greater stability for teachers in colleges. In contrast, factors that were associated 
with a positive impact on retention and achievement, (support for students, 
teaching, personal tutoring and assessment and feedback) are not seen to have 
changed significantly. This is presumably because teachers recognise these 
factors as being an important part of their role over the whole of the 3-year 
period. Both inferences are compatible, and are entirely consistent with the findings 
from the qualitative phase of the research, which are discussed in the second part of 
this report. 
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Responses by category of course/programme 



The next stage of the analysis examined survey responses by the six categories of 
course from high baseline, improving to low baseline, declining. As expected, there 
were different responses from the different categories. Contrary to expectations, 
however, the differences were not very large. Indeed, few of the differences were 
significant at the 5% level. The most unexpected and counter-intuitive finding is that the 
responses that resemble each other most closely are from high baseline, improving 
courses and low baseline, declining courses. The views of respondents from these two 
categories of course can be described as the most ‘optimistic’ or ‘positive’. At the other 
end of the spectrum, the views of staff from courses in the low baseline, improving and 
high baseline, declining categories also resemble each other. 



Table 1 below gives the mean score for each of the 13 types of change given by 
respondents in different categories of course. The changes that have the most 
positive impact on retention and achievement are as follows: 

• context of your course 

• assessment and feedback 

• quality systems 

• support for students 

• teaching 

• personal tutoring 



Table 1: Overall assessment of the impact of 
change factors, by category of course 



Curriculum 

Context of the course/programme 
Assessment and feedback 
Staffing 

Quality systems 
Resources 
Student cohort 
Support for students 
Support for teachers 
Teaching 
Personal tutoring 

Course/programme organisation and administration 
Timetabling 

KEY: 

HI = High baseline, improving 
HM = High baseline, maintaining 
HD = High baseline, declining 



HI 


HM 


HD 


LI 


LM 


LD 


Overall 


3.05 


3.09 


3.01 


2.93 


3.15 


2.99 


3.04 


3.30 


3.13 


2.81 


2.97 


3.24 


3.27 


3.12 


3.25 


3.07 


3.16 


3.05 


3.08 


3.15 


3.13 


3.12 


3.07 


2.95 


3.06 


2.97 


3.24 


3.07 


3.27 


3.16 


3.11 


2.99 


3.16 


3.16 


3.14 


2.81 


2.74 


2.50 


2.59 


2.56 


2.71 


2.65 


2.89 


2.80 


2.58 


2.79 


2.88 


2.91 


2.81 


3.23 


3.23 


3.16 


3.03 


3.24 


3.26 


3.19 


3.06 


2.99 


2.74 


2.89 


2.82 


3.11 


2.94 


3.19 


3.15 


3.00 


2.99 


3.11 


3.08 


3.09 


3.08 


3.27 


3.11 


3.01 


3.24 


3.37 


3.18 


3.08 


2.94 


2.67 


2.87 


3.06 


3.06 


2.95 


3.01 


2.85 


2.86 


2.93 


2.92 


3.12 


2.95 



LI = Low baseline, improving 
LM = Low baseline, maintaining 
LD = Low baseline, declining 
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This is broadly consistent with the more impressionistic, summative judgements made 
by respondents in the previous section of ‘rates of change over the 3-year period’. 
Different patterns of response from different categories of course are also apparent. In 
general, the higher (so more optimistic or positive) ratings of change are from the high 
baseline, improving and the low baseline, declining courses. The lowest ratings are 
from the high baseline, declining and the low baseline, improving courses! 

Similarities and differences in the responses from different categories of course can be 
seen even more clearly in Figure 1. Given that the responses which differed most were 
from high baseline, improving and high baseline, declining courses, the 12 items where 
the difference was greatest were plotted first. The corresponding responses from other 
categories of course were also plotted for further points of comparison. 

Figure 1: How colleges improve 



—©—HI — q — HM — a — HD — w — LI — cf — LM — c — LD 




key: 

B = Context of course/programme D = Staffing F = Resources 

G = Student cohort L = Course/programme organisation and administration 

B7 Whether experiments in your teaching are encouraged 
D1 The membership of your course/programme team 
G1 1 ‘Other’ student cohort factor" 

L8 ‘Other’ course/programme organisation and administration factor 
F7 ‘Other’ resource factor 

B9 ‘Other’ context factor 

F4 Availability of essential equipment for your course/programme 

B5 Morale within your team 

L2 Amount of course administration 

G9 Financial problems of students 

B6 Morale within your department or faculty 



If the original hypotheses had been correct, the low baseline, improving responses 
would have been closer to the high baseline, improving and the low baseline, declining 
responses would have been most similar to those from high baseline, declining 
courses. As the figure makes clear, this is not the case. With the notable exception of 
views of changes to the amount of course administration, responses from low baseline, 
declining courses are quite similar to those from high baseline, improving courses. 
Conversely, across most of the items, the responses from low baseline, improving 
courses are quite similar to those from high baseline, declining courses. These findings 
demand further analysis and provide the focus for the qualitative phase of the research, 
which is discussed in Part 2. 

The changes with the most negative impact on retention and achievement are: 

• resources 

• student cohort 

• support for teachers 

• course organisation and administration 

• timetabling. 

The four changes associated with the least positive/most negative impact broadly 
match the changes identified more impressionistically in the response to the final, 
summative question in the survey. Table 1 also shows the fairly small differences 
separating the different categories of course. Four types of change are rated 
particularly negatively by respondents from high baseline, declining courses: 

• course context 

• student cohort 

• support for teachers 

« course organisation and administration. 

Changes to resources are rated most negatively by respondents from all categories of 
course. Respondents from high baseline, improving courses, however, made the least 
negative assessment of the impact of changes to resources. The most positive 
judgement concerning any changes made by respondents in the low baseline, 
declining category, is in respect of personal tutoring. 
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Findings for specific change factors 



Curriculum 

Curriculum changes that were rated most positively were: 

• scheme of work 

• the way in which the schemes of work are introduced to 
students 

• tutoring arrangements. 

Curriculum changes that were rated least positively were: 

• degree of difficulty of the course 

• arrangements for key skills. 

High baseline, improving courses were particularly positive about changes to the 
scheme of work and the way in which schemes of work were introduced to students. 
The sorts of change in schemes of work which were identified by high baseline, 
improving courses as having a particularly positive impact include: 

• the inclusion of a personal learning journal 

• a large number of enrichment opportunities 

• wider choice in option modules 

• the introduction of exams at the end of the first year 

• replacement of 6-week modules by shorter 3-week modules. 

Table 2 below shows the assessment of the impact of different aspects of curriculum 
change, by different categories of course. 



Table 2: Mean ratings for curriculum factors, 
according to course 





HI 


HM 


HD 


LI 


LM 


LD 


Overall 


Course/syllabus content 


3.05 


3.02 


3.00 


2.81 


3.27 


3.14 


3.05 


Course/programme structure 


2.95 


3.17 


3.12 


2.87 


3.11 


3.13 


3.06 


Degree of difficulty of course/programme for students 


2.67 


2.90 


2.78 


2.87 


2.88 


3.21 


2.89 


Awarding body 


3.05 


3.13 


2.97 


2.96 


3.12 


3.00 


3.04 


Scheme of work 


3.38 


3.06 


3.21 


3.08 


3.12 


3.08 


3.16 


Way in which scheme introduced to students 


3.36 


3.17 


3.18 


3.12 


3.33 


3.00 


3.19 


Amount of enrichment linked to course/programme 


3.15 


3.22 


3.06 


3.08 


3.27 


2.80 


3.10 


Amount of enrichment not linked to course/programme 


2.90 


3.09 


2.84 


2.96 


2.86 


3.14 


2.97 


Tutoring arrangements 


3.15 


3.23 


3.21 


3.06 


3.08 


3.07 


3.13 


Arrangements for key skills 


2.80 


3.00 


2.77 


2.67 


3.26 


2.53 


2.84 


Any other curriculum factor 


3.11 


2.95 


3.00 


2.75 


3.36 


2.82 


3.00 
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KEY: 



HI = High baseline, improving LI = Low baseline, improving 

HM = High baseline, maintaining LM = Low baseline, maintaining 

HD = High baseline, declining LD = Low baseline, declining 



Low baseline, improving courses were least positive about changes to course content 
and ‘other curriculum factors’. ‘Other curriculum factors’ in this context include: 

• the centralisation of timetabling 

• excessive student workload 

• reduction in course hours 

• the tension between 5-year registrations with an awarding body 
and the need for students to complete portfolios within the 2 years 
of their course. 

Low baseline, declining courses rated curriculum changes less positively than they did 
other sorts of changes and were least positive about the impact of changes to the 
amount of enrichment linked to course programme and arrangements for key skills. 
Typical comments included: 

Students were entered for key skills for the last 2 years, which 
has added extra hours to their timetable (and required) and extra 
work for them to complete. 

There has been lack of support ...as the vocational staff are now 
having to deliver, assess and record all the key skill areas (note: 
key skills are a good idea, but should be course specific). 

Context of courses/programmes 

Changes to course context which were rated most positively were: 

• the emphasis placed on improving teaching and learning 

• how successes are dealt with by the team leader 

• how problems are dealt with by the team leader 

• whether innovation in teaching is encouraged 

• quality of internal communications within the teaching 
team. 

Course context changes that were rated least positively were: 

• morale within the team 

• morale within the department or faculty. 
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Table 3 below shows the assessments of the impact of different aspects of changes to 
course context by different categories of course. 



Table 3: Mean ratings for context of 
course/programme factors, according to course 

Quality of internal communications within your team 
Clarity of what is expected of you by your line manager 
How successes are dealt with by course leader 
How problems are dealt with by course leader 
Morale within your team 
Morale within your department or faculty 
Whether innovation in your teaching is encouraged 
Emphasis placed on improving teaching and learning 
Any other context factor 

KEY: 

HI = High baseline, improving 
HM = High baseline, maintaining 
HD = High baseline, declining 



HI 


HM 


HD 


LI 


LM 


LD 


Overall 


3.55 


3.08 


3.17 


3.14 


3.29 


3.60 


3.31 


3.33 


3.06 


2.91 


2.92 


3.21 


3.13 


3.09 


3.50 


3.23 


3.09 


3.06 


3.56 


3.38 


3.30 


3.37 


3.19 


3.18 


3.08 


3.50 


3.38 


3.28 


2.85 


2.91 


2.21 


2.75 


3.14 


3.21 


2.85 


2.75 


2.70 


2.06 


2.48 


2.81 


3.00 


2.63 


3.58 


3.27 


3.03 


3.10 


3.19 


3.43 


3.27 


3.60 


3.44 


3.12 


3.14 


3.37 


3.50 


3.36 


3.17 


3.29 


2.56 


3.05 


3.13 


2.80 


3.00 



LI = Low baseline, improving 
LM = Low baseline, maintaining 
LD = Low baseline, declining 



The least positive assessment of the impact of changes to the course context was 
made by the high baseline, declining courses which also made the most negative 
assessment of the impact of changes to morale and ‘other context factors’. ‘Other 
course context factors’ identified by high baseline, declining courses include: 

• lack of staff development 

• (poor) availability of equipment and accommodation 

• staffing levels 

• deterioration in resourcing 

• fear of closure/redundancy. 

The flavour of the pessimism concerning changes to course context among teachers 
on high baseline, declining courses, can be conveyed by the following comments: 

College morale has been severely damaged, due to restructuring; 
this has communicated itself to students, who have been made 
uncertain of the future. 

Morale within the department is very low. No communication or 
very little, is encouraged. 

It is depressing for my staff to be working in a service which is in 
constant decline, in terms of conditions and ever-escalating 

demands. 
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New or weaker members of teams [are] scorned, or meetings 
held when they cannot easily attend, or they are not informed. 

The most positive assessments of changes to course context were made by the high 
baseline, improving and low baseline, declining courses. The sorts of course context 
changes with the most positive impact were identified as: 

• recognition and appreciation of success 

• encouragement by the head of centre 

• consistency within the team 

• change of course team leader 

• new monitoring procedures 

• more regular team meetings. 

Open-ended comments from high baseline, improving course team members include: 
Success is much more recognised and appreciated. 

Innovation in teaching is always encouraged by the head of 

centre. 

Morale has dropped because of changes in college structure, but 
students have not suffered’. 

The course context change which was identified by respondents from high baseline, 
improving courses as having the most beneficial impact on retention and achievement 
was the emphasis placed on improving teaching and learning. Among the changes to 
course context, emphasis was placed on improving teaching, and respondents from all 
categories viewed learning most optimistically. 

Assessment and feedback 

Changes to assessment and feedback factors that were rated most positively 
were: 



• helpfulness of feedback to students 

• students’ understanding of assessment 

• methods of assessment 

• other assessment/feedback factors. 

The sorts of other changes to assessment/feedback which were identified as having a 
beneficial effect on retention and achievement include: 

• greater clarity concerning standards of assessment 



• standardisation of assignment briefs 

• greater staff familiarity with assessment tasks 

• introduction of a pilot feedback form for use by teachers. 

Some of the evidence is rather ambiguous. Most of the ‘other’ assessment factors 
identified tend to have a negative impact. These include: 



• lack of clear policies for coursework deadlines 

• excessive workload on teachers 

• few assessors 

• delays in returning work to students 

• increases in (staff) workload. 



There were no changes to assessment and feedback factors that were rated 
particularly negatively. The most positive evaluations overall of the impact of changes 
to assessment and feedback were made by high baseline, improving and low baseline, 
declining courses. Examples of particularly positive changes included: 

• improvements to assessment methods after a lecturer became an 
examiner for one of the modules 

• noticeboard displays of work due by the end of each half-term 

• provision of feedback on test papers to students 

• use of standard feedback sheets for all written work by students. 



Table 4 below shows the assessments of the impact of different aspects of changes to 
assessment and feedback, by different categories of course. 



Table 4: Mean ratings for assessment and 
feedback factors, according to course 

Students' understanding of assessment 
Amount of assessment 
Methods of assessment 
Scheduling of assessment 
Policy on deadlines for handing work in 
Amount of private study/homework 
Helpfulness of feedback to students 
Any other assessment/feedback factor 

KEY: 

HI = High baseline, improving 
HM = High baseline, maintaining 
HD = High baseline, declining 



HI 


HM 


HD 


LI 


LM 


LD 


Overall 


3.38 


3.03 


3.18 


3.10 


3.32 


3.09 


3.18 


3.15 


2.75 


3.06 


2.92 


3.08 


2.92 


2.98 


3.33 


3.16 


3.15 


3.10 


3.12 


3.08 


3.16 


3.15 


3.06 


3.06 


3.04 


3.11 


3.00 


3.07 


3.11 


3.06 


3.19 


3.16 


3.00 


3.08 


3.10 


3.00 


2.97 


3.00 


2.84 


2.96 


3.33 


3.02 


3.45 


3.48 


3.38 


3.20 


3.25 


3.33 


3.35 


3.40 


3.07 


3.24 


3.00 


2.83 


3.33 


3.15 



LI = Low baseline, improving 
LM = Low baseline, maintaining 
LD = Low baseline, declining 
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Staffing 

The changes to staffing which were rated most positively were: 



• mix of skills in the team 

• course/team leadership 

• membership of the team. 

Staffing changes which were rated most negatively were: 

• any other staffing factor 

• amount of staff absence 

• senior curriculum leadership. 



‘Other staffing factors’ in this context include: 

• a large influx of part-time staff 

• increases in staff workload 

• staff turnover leading to inconsistent practices and a lack of 
(student) confidence in some tutors 

• large and unwieldy course teams 

• difficulties in recruiting staff with the necessary experience and 
abilities. 

The words of one respondent are quite representative of the sorts of views expressed: 

Statistics show that where staff are ill for long periods without 
adequate cover, morale and work suffer amongst students. A high 
percentage of part-time staff find it difficult to fully integrate into 
workings of department. 

The general picture that emerges from views of changes to staffing is of a rather 
beleaguered profession, which is confident about its crucial contribution to retention 
and achievement, but beset by difficulties. The spectrum of views from the most to the 
least optimistic can be illustrated by the following comments: 

Very skilled, well-qualified teachers are able to stretch higher 
ability students [leading to] better further maths results. 

Stability of science team and a single tutor for students improved 
communication between staff and students. 
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One member of staff was replaced by someone with more energy 
and enthusiasm and [....] this made an impact on the interest 
level of the students. 



Considerable problems due to unsuitable staff, long-term sickness 
of one member and redundancy of another. 

All the staff are now part-time. There is no day when we are all in. 
Communication is not straightforward. Second years [. . .] have no 
one to pull their grades together. 



Table 5 shows the assessments of the impact of different aspects of staffing change, 
by different categories of course. 



Table 5: Mean ratings for staffing, 
according to course 



Membership of your course/programme team 

Mix of skills in your course/programme team 

Your course/programme team leadership 

Ratio of full- to part-time teachers in your course team 

Amount of staff absence in your team 

Your department/faculty headship 

Senior curriculum leadership in your college 

Composition of the group of personal tutors 

who tutor your students 

Any other staffing factor 



HI 


HM 


HD 


LI 


LM 


LD 


Overall 


3.48 


3.21 


2.94 


3.14 


3.03 


3.27 


3.18 


3.57 


3.29 


3.09 


3.14 


3.33 


3.13 


3.26 


3.33 


3.28 


3.13 


3.16 


3.41 


3.29 


3.27 


2.90 


3.00 


2.94 


3.10 


2.83 


3.00 


2.96 


2.95 


2.95 


2.88 


2.86 


2.75 


3.29 


2.95 


3.05 


2.85 


3.07 


3.02 


2.85 


3.36 


3.03 


2.86 


2.95 


2.81 


2.94 


2.82 


3.29 


2.95 


2.94 


3.13 


3.03 


3.21 


3.07 


3.27 


3.11 


3.00 


2.95 


2.64 


2.94 


2.64 


3.25 


2.90 



KEY: 



HI = High baseline, improving 
HM = High baseline, maintaining 
HD = High baseline, declining 



LI = Low baseline, improving 
LM = Low baseline, maintaining 
LD = Low baseline, declining 



Quality systems 

The only change to quality systems which was rated negatively was: 

• availability of information from management information 
systems (MIS). 

The change to quality systems that was rated most positively was: 

• the use you make of feedback from students. 
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Low baseline, declining courses gave a particularly negative evaluation of the impact of 
the availability of information from MIS. There was a statistically significant difference 
(at the 5% level) between the response of high baseline, maintenance courses and low 
baseline, improving courses, in respect of changes to observation of teaching and 
learning. Positive comments made by high baseline, maintenance courses included: 

...much more observation of teaching... significant positive impact 
on teaching - some part-time staff identified as requiring further 
training and advice... 

Teaching observations are completed annually and I personally 
feel this has a positive impact on the teaching methods within the 
college and the department. 

In contrast, one respondent from a low baseline, improving course noted the 
‘introduction of a failed system for classroom observations’. Another remarked that 
‘increased “inspection” for senior lecturers has a demotivating effect - nothing positive 
is achieved by it’. 

Table 6 shows the assessments of the impact of different aspects of change to quality 
systems, by different categories of course. 



Table 6: Mean ratings for quality systems, 
according to course 



Self-assessment process for your course/programme 
Availability of information from MIS 
Observation of teaching and learning 
Observation of tutoring 

Processes for generating feedback from students 
Use you make of feedback from students 
Ways that managers monitor performance 
on your course/programme 
Any other quality systems factor 

KEY: 

HI = High baseline, improving 
HM = High baseline, maintaining 
HD = High baseline, declining 



HI 


HM 


HD 


LI 


LM 


LD 


Overall 


3.25 


3.15 


3.09 


3.08 


3.31 


3.07 


3.16 


3.40 


2.92 


2.97 


2.88 


3.19 


2.77 


3.02 


3.30 


3.30 


3.29 


2.94 


3.11 


3.13 


3.18 


3.17 


3.19 


3.12 


2.86 


3.14 


3.14 


3.10 


3.10 


3.20 


3.25 


3.08 


3.18 


3.21 


3.17 


3.37 


3.37 


3.33 


3.12 


3.28 


3.36 


3.31 


3.33 


3.00 


2.88 


2.96 


3.03 


3.29 


3.08 


3.25 


3.14 


2.94 


3.00 


3.00 


3.33 


3.11 



LI = Low baseline, improving 
LM = Low baseline, maintaining 
LD = Low baseline, declining 
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Resources 



All aspects of changes to resources were evaluated negatively. The most negative 
evaluation concern: 



• time you have available for preparation 

• number of teaching hours you have 

• availability of essential equipment 

• availability of technical support and backup. 

Particularly negative evaluations were made concerning the time available for 
preparation and the number of teaching hours. Typical comments include: 

limited time for getting lesson ready sometimes means fewer 
notes/materials prepared 

...time for lesson preparation work is reduced, due to teaching 
hours/other paperwork/administration. This reflects on the number 
of different courses (taught by individual teachers). 

I work 60/70 hours a week and still never seem to keep ahead. 

High baseline, declining courses made notably more negative evaluations than other 
categories of course in respect of time available for preparation and the availability of 
essential equipment. A selection of comments from such courses gives the impression 
of something approaching despair: 

Do not feel I have enough time to prepare lessons to the best of 
my ability, due to mountains of repetitive paperwork. 

I’m working harder than ever, but am less prepared for all my 
lessons - students need good lessons. 

Changed job role has cut time available for preparation/marking. 

New contracts exacerbate a situation. Now teaching more hours 
(with 5 hours remission) than 5 years ago! 
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Table 7 shows the assessments of the impact of different aspects of change to 
resources, by category of course. 



Table 7: Mean ratings for resources, 
according to course 





HI 


HM 


HD 


LI 


LM 


LD 


Overall 


Time you have available for preparation 


2.43 


2.33 


2.18 


2.36 


2.35 


2.46 


2.35 


Number of teaching hours you have 


2.61 


2.67 


2.48 


2.61 


2.48 


2.36 


2.54 


Number of different courses you teach on 
Availability of essential equipment for your 


2.86 


2.73 


2.64 


2.65 


2.88 


2.62 


2.73 


course/programme 


2.90 


2.78 


2.38 


2.55 


2.50 


2.77 


2.65 


Availability of technical support and back up 


2.86 


2.81 


2.57 


2.46 


2.52 


3.00 


2.70 


Accommodation for your course/programme 


2.85 


2.83 


2.77 


2.76 


2.72 


2.77 


2.78 


Any other resource factor 


3.17 


3.06 


2.50 


2.71 


2.45 


3.00 


2.82 



KEY: 



HI = High baseline, improving 
HM = High baseline, maintaining 
HD = High baseline, declining 



LI = Low baseline, improving 
LM = Low baseline, maintaining 
LD = Low baseline, declining 



Student cohort 

All aspects of change to the student cohort were also evaluated negatively, with 
the exception of ‘other factors’. The least negative evaluations concern changes to 
the clarity of personal/career goals of students and the proportion of students leaving 
for a job. The most negative evaluations were made in respect of changes in the: 

• amount of part-time work done by students 

• financial problems of students 

• personal/emotional/health problems of students 

• learning skills of students 

• ability of students. 

High baseline, declining courses made some of the most negative evaluations of 
changes to the student cohort, particularly in respect of: 

• financial problems 

• amount of part-time work 

• personal/emotional/health problems. 

Two comments from teachers on high baseline, declining courses are typical of the 
rather pessimistic assessment of the impact of changes to the student cohort: 
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Students come ill prepared for a course which demands a high 
degree of self-motivation, lateral thinking, commitment and 
determination. A significant shift from students with A-levels to 
GNVQ qualifications. Students seem to have far more problems, 
financial, emotional, health than a few years ago; these 
adversely affect attendance, motivation and the quality of 
materials that students are able to purchase, and consequently 
the quality of work they produce. 

Overall, students are less motivated to complete prescriptive 
coursework. They do not find study easy and although they 
have the same number of [GCSE] points, they do not seem so 
academically able. Many more need learning support and spend 
a lot of free time at part-time work. Many more have personal 
problems than previously. 

Table 8 shows the assessments of the impact of different aspects of change to the 

student cohort, by category of course. 



Table 8: Mean ratings for student cohort, 
according to course 





HI 


HM 


HD 


LI 


LM 


LD 


Overall 


Motivation of students on your course/programme 


2.95 


2.84 


2.69 


2.67 


2.81 


3.00 


2.83 


Ability of students 

Qualifications of students at commencement of 


2.67 


2.84 


2.69 


2.83 


2.74 


2.77 


2.76 


course/programme 


2.90 


2.89 


2.74 


2.85 


2.69 


2.85 


2.82 


Learning skills of students 


2.70 


2.81 


2.60 


2.77 


2.79 


2.75 


2.74 


Clarity of personal/career goals of students 


3.00 


3.02 


2.97 


2.83 


2.96 


3.00 


2.96 


Proportion of students leaving for a job 


3.14 


2.97 


2.73 


2.84 


3.11 


3.08 


2.98 


Amount of paid part-time work done by students 


2.73 


2.54 


2.19 


2.60 


2.89 


2.54 


2.58 


How near to college do students live 


2.82 


2.79 


2.67 


2.92 


2.96 


2.92 


2.85 


Financial problems of students 


2.76 


2.58 


2.06 


2.77 


2.65 


2.93 


2.63 


Personal/emotional/health problems of students 


2.68 


2.59 


2.28 


2.62 


2.70 


3.00 


2.65 


Any other student cohort factor 


3.38 


2.91 


2.78 


2.96 


3.33 


3.20 


3.09 



KEY: 



HI = High baseline, improving 
HM = High baseline, maintaining 
HD = High baseline, declining 



LI = Low baseline, improving 
LM = Low baseline, maintaining 
LD = Low baseline, declining 



Support for students 



No changes associated with support for students were evaluated as having a 
negative impact The changes that were evaluated most positively were: 

• amount of support available to ‘at-risk’ students 

• counselling and guidance support available 

• ways that students are encouraged to support each other 

• integration of learning support. 

Some of the most positive evaluations were made by the low baseline, declining 
courses in respect of the involvement of parents of younger students, financial support 
available and counselling and guidance support available. 

The comments provided by a number of respondents helped to illustrate those aspects 
of support for students that seemed to be most effective. 

Interview process for student intake has improved considerably. 

This identifies students with difficulties/problems. Also, 
observations during induction have helped to highlight student 

problems. 

[support for students] has been affected favourably by the 
implementation of senior tutor [role], identified as being 
responsible of student support. 

Parents invited to parents’ evenings; tutors phone/write to parents 
if student is a cause for concern. Introduction of [Educational 
Maintenance Allowance] EMA has helped students financially but 
some still struggle. More students keen to get help - those on 
EMA have to get help. 
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Table 9 shows the assessments of the impact of different aspects of change to support 
for students, by category of course. 



Table 9: Mean ratings for support for students, 
according to course 



Involvement of parents of younger students 
Ways that students are encouraged to support 
each other 

Financial support available 
Counselling and guidance support available 
Availability of staff to provide individual support 
when needed 

Take-up of learning support by students 
Integration of learning support within your 
course/programme 

Length of time it takes to identify 'at-risk' students 
Amount of support available for 'at-risk' students 
Methods of acknowledging/rewarding student progress 
Any other support for students factor 

KEY: 

HI = High baseline, improving 
HM = High baseline, maintaining 
HD = High baseline, declining 



HI 


HM 


HD 


LI 


LM 


LD 


Overall 


2.89 


3.11 


3.17 


3.04 


3.00 


3.31 


3.09 


3.35 


3.30 


3.00 


3.14 


3.32 


3.33 


3.24 


3.26 


3.10 


3.00 


3.18 


3.39 


3.54 


3.19 


3.42 


3.27 


3.23 


3.06 


3.31 


3.46 


3.25 


3.10 


3.34 


3.23 


2.86 


3.26 


3.23 


3.17 


3.15 


3.15 


3.37 


2.84 


3.21 


2.92 


3.11 


3.14 


3.31 


3.27 


3.04 


3.41 


3.07 


3.21 


3.50 


3.11 


3.16 


3.02 


3.30 


3.29 


3.23 


3.40 


3.31 


3.24 


3.10 


3.33 


3.29 


3.28 


3.35 


3.14 


3.28 


3.10 


3.04 


3.23 


3.19 


3.00 


3.36 


2.82 


2.94 


3.11 


3.17 


3.07 



LI = Low baseline, improving 
LM = Low baseline, maintaining 
LD = Low baseline, declining 
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Support for teachers 



No aspects of changes to support for teachers were evaluated as having a 
particularly positive impact on student retention and achievement Changes to 
support for teachers that were evaluated particularly negatively were: 

• mechanisms to acknowledge/reward effective teaching 

• support available for you from within the college 

• administrative/clerical support. 

Some of the most negative evaluations were made by respondents from high baseline, 
declining courses, about: 

• support available to you from within your college 

• mechanisms to acknowledge/reward effective teaching 

• support available to you from within your team. 

Even at their most positive, teachers on high baseline, declining courses appear to feel 
unsupported and under-valued. At their most negative, they feel undermined and even 
disparaged. This range of views can be illustrated by the following comments from the 
questionnaires: 

The college staff do their best to offer support under difficult 
conditions, ie short staffing in areas such as counselling or 

administration. 

Support is just about non-existent!! 

I don’t feel supported by central management - I feel disparaged. 

In many ways, they convey [the view] that we are not valued. 

I feel that any initiative or something slightly out of the norm is 
seen as a problem administrators cannot cope with! 

Support is subject to various interpretations. It is present if you 
tow the line and do not put forward new ideas. 

The most positive evaluation of any aspect of support for teachers was made by low 
baseline, declining courses in respect of support available to you from within your team. 
The most favourable evaluations overall of changes to support for teachers were made 
by low baseline, declining courses and high baseline, improving courses. 

Without wanting to push the interpretation too far, something different seems to be 
happening in different categories of course. A decline in performance in the high 
baseline courses seems to be associated with some disruption in relationships, either 
within the team or between the team and the college. The relatively high rating for team 
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support among teachers on low baseline, declining courses may perhaps indicate that 
such teams support each other in diversity. The positive (but not strongly positive) 
evaluations of team and departmental support from high baseline, improving courses 
may indicate that members of such teams either take their team support for granted, or, 
alternatively that they are less dependent on it. 

These assessments of the impact of changes to support for teachers have some quite 
worrying implications for college leadership. It is, perhaps, natural and unsurprising that 
teachers should have the most positive view of support from their teaching teams. It is, 
however, rather alarming that they should have negative views of the impact of 
changes to support available from their college, or from mechanisms to acknowledge 
and reward effective teaching. 



Table 10 shows the assessments of the impact of different aspects of change to 
support for teachers, by category of course. 



Table 10: Mean ratings for support for teachers, 
according to course 



Support available to you from within your team 

Support available to you from within dept or faculty 

Support available to you from college 

Mechanisms to acknowledge/reward effective teaching 

Opportunities available for staff development 

Administrative/clerical support 

Any other support for teachers factor 



HI 


HM 


HD 


LI 


LM 


LD 


Overall 


3.05 


3.14 


2.79 


3.30 


3.15 


3.50 


3.16 


3.14 


3.03 


2.83 


3.08 


2.79 


3.17 


3.01 


2.90 


2.84 


2.52 


2.68 


2.89 


2.92 


2.79 


2.90 


2.72 


2.52 


2.68 


2.77 


2.92 


2.75 


3.29 


3.12 


2.88 


2.90 


3.13 


3.31 


3.11 


3.00 


2.98 


2.77 


2.74 


2.69 


2.93 


2.85 


3.17 


3.07 


2.90 


2.83 


2.30 


3.00 


2.88 



KEY: 



HI = High baseline, improving 
HM = High baseline, maintaining 
HD = High baseline, declining 



LI = Low baseline, improving 
LM = Low baseline, maintaining 
LD = Low baseline, declining 



Teaching 

Changes to aspects of teaching which were said to have the most positive 
impact on student outcomes were: 

• variety of teaching strategies 

• ways you monitor individual student progress 

• induction processes. 

The change said to have the most negative effect, was to the amount of teaching 
time for work with individual students. 
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Table 11, below, shows the assessments of the impact of different aspects of change 
to teaching, by category of course. 



Table 1 1 : Mean ratings for teaching, 
according to course 



Your role as a teacher 
Recruitment and selection processes 
Entry requirements/admissions policy 
Induction processes 

Ways you monitor individual student progress 

Variety of teaching strategies you use 

Amount of teaching time involving different tasks for lower 

(or higher) attaining students 

Amount of teaching time available to work with 

individual students 

Proportion of teaching time involving 'active learning' 
by students 

Your involvement in development activity inside 
your college 

Your involvement in professional activity outside 
the college 

Any other teaching factor 



HI 


HM 


HD 


LI 


LM 


LD 


Overall 


3.14 


3.14 


2.75 


2.82 


3.21 


2.71 


2.96 


3.23 


3.22 


3.13 


3.18 


3.00 


3.29 


3.18 


3.19 


3.00 


3.03 


3.08 


3.00 


3.21 


3.09 


3.39 


3.33 


3.22 


3.16 


3.35 


3.21 


3.28 


3.48 


3.32 


3.33 


3.10 


3.45 


3.46 


3.36 


3.52 


3.42 


3.20 


3.12 


3.32 


3.54 


3.35 


3.20 


3.14 


2.73 


2.94 


2.89 


2.86 


2.96 


2.70 


2.75 


2.45 


2.62 


3.00 


2.79 


2.72 


3.19 


3.17 


3.07 


2.98 


3.25 


2.86 


3.09 


3.10 


3.02 


3.13 


2.96 


3.07 


3.00 


3.05 


3.15 


3.26 


3.16 


2.92 


3.04 


2.79 


3.05 


3.00 


3.00 


2.80 


3.00 


2.78 


3.25 


2.97 



KEY: 



HI = High baseline, improving LI = Low baseline, improving 

HM = High baseline, maintaining LM = Low baseline, maintaining 

HD = High baseline, declining LD = Low baseline, declining 



The free text comments from the questionnaires add considerably to this rather bare 
analysis. It is clear, from a number of such comments, that changes to 
recruitment/selection and induction processes are generally reviewed as having a 
positive effect. Further, a number of teachers convey a sense of satisfaction and pride 
in their professional development and, indeed, advancement. These are viewed as 
having helped teachers better to assist their students: 

The contribution to developing activity, both inside college and the 
opportunity to develop my own professional life outside, has had a 
positive impact on the course. 

Promotion to a management post means I feel I can be more 
helpful to students in my own right and I like to speak to groups of 
students I teach to discuss problems/positive issues. 
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[Morale within my team] has changed dramatically: 3 years ago I 
was an instructor but now I am a full-time lecturer [and 
programme manager] [. . .] after completing a Cert. Ed. 

...1 year teaching experience in the USA has helped to improve 
teaching skills and techniques. 

Involvement with exam/awarding body has had a significant 
positive impact. 

At the other end of the spectrum, however, comments concerning the reduction of 
teaching time available to work with individual students resemble almost exactly the 
comments made on resources, discussed above. 

Removal of workshop time has made it increasingly difficult to 
help students with individual problems. 

Teaching time in the college is cut for this course annually. As a 
result, some of the weaker students will suffer or leave. 

Size of teaching groups has increased, resulting in a more 
uniform approach to teaching strategies and less time for 
differentiation or individual support. 

Less time for dealing with individual needs of students and 
similarly ‘group work’ involves whole course, and all are treated in 
a similar way. No time for applying teaching styles. 

Spending too much time with weak students is throwing the group 
off balance and not enabling enough time for more able students. 

Overall, the high baseline, improving courses evaluated changes to teaching 
most positively. At the other extreme, low baseline, improving courses were the 
least optimistic concerning the impact of changes to teaching. 

Personal tutoring 

All aspects of changes to personal tutoring were evaluated as having a generally 
positive impact on retention and achievement Changes to the aspects of 
tutoring which were rated most positively were: 

• any other personal tutoring factor 

• the role of the personal tutor 

• target-setting processes in personal tutorials 

• review and action-planning processes in personal tutorials. 
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Other personal tutoring factors in this context include: 

• support for basic skills and special needs 

• interaction with parents 

• support for personal, social and emotional problems of students 

• improved tutorial framework. 

Typical positive comments from the questionnaires include: 

Much more time allocated with much improved response [...] 
better reviewing of action plans during tutorials has improved 
tutorials and retention. 

Personal tutor extremely good in her role - very supportive yet 
realistic with students. 

The reviewing has been excellent and really is useful for target 

setting. 

Tutorial provision acts as safety net to retain students on the 
programme - they can be directed to suitable support. 

The personal tutor process has enabled issues to be resolved 
early - enhancing retention figures. 

Changes to tutoring were evaluated most positively by respondents from low baseline, 
declining courses. As we have seen, in relation to other change factors, high baseline, 
improving courses tended to give some of the most positive evaluations to other types 
of change. In relation to personal tutoring, however, they joined the low baseline, 
improving courses in evaluating changes to personal tutoring in a fairly neutral way. 

The main limiting factor associated with personal tutoring is lack of sufficient time. 
Where personal tutoring attracts negative assessments, this is invariably associated 
with this problem, or (more rarely) with an inappropriate focus on paperwork and 
administration. The following comments are fairly typical: 

The key issue here is the amount of time [. . .] Other issues, such 
as increased teaching load, are significant here. 

The role [of personal tutor] remains very important. The crap that 
goes with it (such as target setting, action planning, etc) puts 
students off and gets in the way of ‘proper tutoring’. Let us be 
effective front-line tutors! Why is everything so paper-bound? 

Table 12 shows the assessments of the impact of aspects of change to personal 
tutoring, by category of course. 
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Table 12: Mean ratings for personal tutoring 
(on this course/programme), according to course 

Role of the personal tutor 

Amount of time students have for one-to-one tutorials 

Amount of time students have for group tutorials 

Target setting processes in personal tutorials 

Review and action-planning processes 

in personal tutorials 

Any other personal tutoring factor 

KEY: 

HI = High baseline, improving 
HM = High baseline, maintaining 
HD = High baseline, declining 



HI 


HM 


HD 


LI 


LM 


LD 


Overall 


3.00 


3.30 


3.10 


3.02 


3.54 


3.36 


3.22 


2.86 


3.15 


3.07 


2.87 


3.16 


3.43 


3.09 


3.05 


3.21 


2.94 


2.92 


3.15 


3.36 


3.11 


3.14 


3.26 


3.27 


3.10 


3.19 


3.36 


3.22 


3.05 


3.27 


3.30 


3.08 


3.23 


3.36 


3.22 


3.38 


3.43 


3.00 


3.06 


3.18 


3.33 


3.23 



LI = Low baseline, improving 
LM = Low baseline, maintaining 
LD = Low baseline, declining 



Course organisation and administration 

Overall, changes to course organisation and administration were not seen as 
having a particularly positive impact. The change that was evaluated most positively 
was the ways you monitor and record student progress. The changes that were rated 
most negatively were the amount of course administration and time available for team 
meetings. It is interesting to note that the high baseline, improving courses gave the 
most positive assessment of changes to course organisation and administration. 
Qualitative comments from their questionnaires included the following: 

Course administration has greatly increased, however it has been 
beneficial to us. 

Shared learning of effective delivery of course, [and] recording 
student progress has had a positive impact. 
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Table 13 shows the assessments of the impact of different changes to course 
administration and organisation, by category of course. 



Table 13: Mean ratings for course/programme 
organisation and administration, according to 
course 



Ways you monitor and record student progress 

Amount of course administration 

Time available for team meetings 

Effectiveness of team meetings 

Team use of shared learning materials 

Team involvement in planning all aspects of course 

Amount of team teaching 

Any other course/programme organisation 

and administration factor 



HI 


HM 


HD 


LI 


LM 


LD 


Overall 


3.26 


3.27 


3.06 


3.14 


3.50 


3.15 


3.23 


2.84 


2.55 


2.21 


2.55 


2.59 


2.29 


2.51 


2.71 


2.56 


2.30 


2.62 


2.93 


3.00 


2.69 


3.11 


2.92 


2.70 


2.82 


3.12 


3.13 


2.97 


3.17 


3.18 


2.97 


3.02 


3.19 


3.36 


3.15 


3.21 


3.12 


2.81 


2.96 


3.31 


3.36 


3.13 


3.00 


2.95 


2.69 


2.87 


3.00 


3.15 


2.94 


3.33 


3.00 


2.64 


2.94 


2.83 


3.00 


2.96 



KEY: 



HI = High baseline, improving 
HM = High baseline, maintaining 
HD = High baseline, declining 



LI = Low baseline, improving 
LM = Low baseline, maintaining 
LD = Low baseline, declining 



Timetabling 

With some exceptions, changes to timetabling were seen to have a negative 
impact on student outcomes. The changes that were evaluated as having the 
most negative effect were: 

• hours within which the courses were taught each day 

• amount of gaps in the timetable 

• amount of timetabled private study by students 

• amount of time on the timetable for students to ‘catch up’. 

In respect of four of the six aspects of changes to timetabling, and in respect of the 
overall assessment of changes to timetabling, respondents from low baseline, declining 
courses gave the most optimistic assessment of their impact on retention and 
achievement. 
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Table 14, below, shows the assessments 
timetabling, by category of course. 

Table 14: Mean ratings for timetabling, 
according to course 

Hours within which the course/programme is taught 
each day 

Amount of gaps in timetable 

Number of weeks the course is taught over the year 
Amount of timetabled private study time for students 
Amount of time on timetable for students to 'catch up' 

Any other timetabling factor 

KEY: 

HI = High baseline, improving 
HM = High baseline, maintaining 
HD = High baseline, declining 



of the impact of aspects of change to 



HI 


HM 


HD 


LI 


LM 


LD 


Overall 


2.90 


2.97 


2.83 


2.94 


2.79 


2.73 


2.86 


2.85 


2.72 


2.74 


2.98 


2.97 


3.27 


2.92 


3.16 


2.95 


3.10 


2.94 


2.97 


2.86 


3.00 


3.00 


2.85 


2.79 


2.91 


2.87 


3.14 


2.93 


2.95 


2.80 


2.83 


2.85 


3.14 


3.21 


2.96 


3.17 


2.79 


2.86 


2.94 


2.78 


3.50 


3.01 



LI = Low baseline, improving 
LM = Low baseline, maintaining 
LD = Low baseline, declining 



Discussion: comparisons between categories of course 

High baseline courses 

Respondents from high baseline courses seem to make broadly similar assessments of 
the sort of changes that have the most positive and most negative impact on student 
outcomes. As originally anticipated, the aspects of the student experience which are 
most closely controlled and influenced by teachers (innovation, team skills, emphasis 
on improving teaching and learning), are seen as having broadly positive impacts. 
Aspects that are not controlled by teachers (student characteristics, morale within the 
department, the degree of difficulty of the course) are seen as having largely negative 
impacts on retention and achievement. In addition, changes to factors which have the 
effect of constraining or reducing the ability of teachers to teach effectively (time for 
preparation, amount of course administration, number of teaching hours) are seen as 
reducing achievement and retention rates. 

A further aspect of the data is striking and corresponds with our initial hypotheses. 
Respondents from high baseline, improving courses are the most positive about 
positive changes, eg emphasis placed on improving teaching and learning and the 
least negative about changes with broadly negative impacts, eg time you have for 
preparation. The opposite is true for the respondents from high baseline, declining 
courses who are least positive about the changes with positive impacts, eg helpfulness 
of feedback to students and most negative about changes with negative impacts, eg 
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morale within your department or faculty. Responses from high baseline courses which 
are maintaining their retention and achievement rates fall somewhere between the two. 
This point can be demonstrated by reference back to the overall evaluations of different 
change factors and to the fairly consistent gap in responses from improving and 
declining, high baseline courses (see pages19-37). 

The same point can be demonstrated in a different way. The five most positive and the 
five most negative scores for any aspect of change are listed below, along with the 
mean score for each item. The high baseline, improving courses (ie the most positive) 
gave the highest scores for the most positive items and also the highest (ie the least 
negative) scores for the items that were seen as having the most adverse effect on 
retention and achievement. At the other end of the spectrum, high baseline, declining 
courses gave the most pessimistic (ie lowest) assessments; high baseline courses, 
which were maintaining their success rates were somewhere in between. 

There is an indication, moreover, that respondents from high baseline, improving 
courses are particularly positive about the emphasis on improving teaching and 
learning, the encouragement of innovation and some aspects of the ways in which their 
teaching teams function. 



High baseline, improving courses 



The changes which respondents from high baseline, improving courses viewed most 
positively are listed below. The figures in brackets give the mean score: 



• emphasis placed on improving teaching and learning (3.60) 

• whether innovation in teaching is encouraged (3.58) 

• mix of skills in your course team (3.57) 

• quality of internal communications in your team (3.55) 

• variety of teaching strategies you use (3.52). 



The changes that are seen to have the most negative impact on retention and 
achievement are: 



• time you have available for preparation (2.43) 

• number of teaching hours you have (2.61 ) 

• ability of students (2.67) 

• degree of difficulty of the course for students (2.67) 

• personal/emotional/health problems of students (2.68). 
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High baseline, maintaining courses 

The changes that respondents from high baseline, maintaining courses identified as 
having the most beneficial impact are: 

• helpfulness of feedback to students (3.48) 

• emphasis placed on improving teaching and learning (3.44) 

• other personal tutoring factors (3.43) 

• variety of teaching strategies you use (3.42) 

• use you make of feedback from students (3.37). 

The changes that the same respondents said had the most negative impact are: 

• time you have available for preparation (2.33) 

• amount of paid part-time work done by students (2.54) 

• amount of course administration (2.55) 

• time available for team meetings (2.56) 

• financial problems of students (2.58). 

High baseline, declining courses 

The changes with the most positive impact on retention and achievement according to 
respondents from high baseline, declining courses are: 

• helpfulness of feedback to students (3.38) 

• take up of learning support by students (3.37) 

• use you make of feedback from students (3.33) 

• ways you monitor individual student progress (3.33) 

• review and action-planning processes in personal tutorials (3.30). 

The changes with the most negative impact, according to respondents from high 
baseline, declining courses are: 

• morale within your department or faculty (2.06) 

• financial problems of students (2.06) 

• time you have available for preparation (2.18) 

• amount of part-time work done by students (2.19) 

• amount of course administration (2.21 ). 
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Low baseline courses 



The most surprising feature of the low baseline courses is that respondents from low 
baseline, improving courses are much less positive than respondents from courses that 
have only maintained their success rates and respondents from low baseline courses 
whose success rates have actually deteriorated. In the 13 sections of the survey, they 
gave the least or the next least optimistic assessment of the impact of changes in nine 
categories. Their ‘pessimism’ was exceeded only by respondents from high baseline, 
declining courses who gave the least or the next to least optimistic evaluation of the 
impact of 1 1 categories of change. This point is worth emphasising in a different way. 
What distinguishes improving low baseline courses is their relative pessimism 
concerning the impact of changes that are regarded as having a beneficial impact on 
retention and achievement. Their evaluation of the impact of changes that are generally 
regarded as negative is not markedly different from that of other categories of course. 

The second surprising feature of this data is that respondents from low baseline 
courses with declining success rates seem to have a view of positive change factors 
which does not readily accord with data concerning trends and success rates on their 
own courses. Thus, respondents from low baseline, declining courses are noticeably 
more positive in their assessment of change factors with a generally favourable impact 
on achievement and retention than respondents from low baseline improving courses. 
Indeed, their assessments quite closely resemble the assessments made by the high 
baseline, improving courses. In the 13 categories of change, respondents from low 
baseline, declining courses gave the most or next most optimistic evaluation of eight 
categories. This general observation can be illustrated by reference to the scores given 
by different categories of low baseline courses to the most positive and most negative 
change factors. The scores for the most negative factors are quite similar between the 
different categories of low baseline courses. The scores for the most positive factors, 
however, are higher from the low baseline maintaining or declining courses and 
paradoxically, lower from the low baseline, improving courses. 

Low baseline, improving courses 

The changes which respondents from low baseline, improving courses viewed most 
positively are listed below: 

• support available to you from within your team (3.30) 

• composition of the group of personal tutors who tutor your students 
(3.21) 

• helpfulness of feedback to students (3.20) 

• recruitment and selection processes (3.18) 

• financial support available (3.18). 
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The changes that are assessed as having the most negative impact are listed below: 

• time you have available for preparation (2.36) 

• availability of technical support and backup (2.46) 

• morale within your department or faculty (2.48) 

• availability of essential equipment (2.55) 

• amount of course administration (2.55). 

Low baseline, maintenance courses 

The changes seen most positively by respondents from low baseline courses whose 
success rates were maintained are: 

• how successes are dealt with by your course leader (3.56) 

• the role of the personal tutor (3.54) 

• how problems are dealt with by your course leader (3.50) 

• ways you monitor and record student progress (3.50) 

• ways you monitor individual student progress (3.45). 

The changes that the same respondents evaluated as having the most negative impact 
on retention and achievement are all related to resources: 

• any other support for teachers (2.30) 

• time you have available for preparation (2.35) 

• any other resource factor (2.45) 

• number of teaching hours you have (2.48) 

• availability of essential equipment (2.50). 



Low baseline, declining courses 

Contrary to expectation, respondents from courses with a low baseline and declining 
success rates evaluated a number of strategies as having a strong positive impact on 
retention and achievement. The changes whose impact was assessed most positively 
are: 



• quality of internal communications in your team (3.60) 

• financial support available to students (3.54) 

• variety of teaching strategies you use (3.54) 

• emphasis placed on improving teaching and learning (3.50) 

• support available to you from within your team (3.50). 
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The changes said to have the most negative impact are: 

• amount of course administration (2.29) 

• number of teaching hours you have (2.36) 

• time available for preparation (2.46) 

• arrangements for key skills (2.53) 

• amount of paid part-time work done by students (2.54). 



Summary 

The biggest surprise from the quantitative phase of this research was the relatively 
small difference in views expressed between the different categories of course. Not 
only were the differences fairly small, but they tended not be statistically significant, at 
the 5% and even at the 10% level. This implies a certain shared professional outlook 
across this relatively large, diverse and structured sample of experienced teachers, 
tutors and course team leaders. 

Factors that are seen as having a positive effect on student retention and achievement 
rates are mainly those factors that teachers can control or influence: 

• support for students 

• quality systems 

• personal tutoring 

• assessment and feedback 

• context of your course 

• teaching. 

Factors that are seen as having an adverse impact on student retention and 
achievement are also identified as those factors which changed most significantly over 
the 3-year period. These are, generally, factors over which teachers have least 
influence or control: 

• resources 

• student cohort 

• support for teachers 

• course organisation or administration 

• timetabling. 

A second finding is that the most marked differences between the responses from the 
different categories of course were manifested in the responses from high baseline, 
improving and high baseline, declining courses. The former were most positive about 
the factors that teachers control and least negative about the factors which lay outside 
teachers’ control. The latter were least positive/most negative about both. This finding 
largely accords with the original hypotheses. 
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The third finding was, to say the least, counter-intuitive. Respondents from low 
baseline, declining courses tended to express views that were quite similar to those 
expressed by teachers from the high baseline, improving courses. Conversely, the 
views of teachers on low baseline, improving courses seemed to be quite pessimistic, 
particularly where their views are compared with the views of respondents from 
courses with the same low baseline, but which had actually declined over the 3-year 
period. 

Various hypotheses were suggested in Steering Group discussions. Perhaps the 
teachers on these low baseline, declining programmes were not fully aware of their 
data? Could their data be wrong? Were teachers comparing their performance to some 
sort of norm for their particular subject or programme area, rather than the college 
norm? Did teachers on low baseline, declining courses feel vulnerable and did this 
colour the way that they completed questionnaires? In other words, did they give the 
answers which they thought might be expected of them, rather than the answers which 
represented their honest views. In any event, the second phase of the research sought 
to answer these questions, to clarify the questionnaire responses, to gather further 
information and, above all, to clarify understandings and interpretations of the survey 
phase of this research. 
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PART 2: QUALITATIVE RESEARCH 



Introduction 

The interviews were conducted between September and December 2001. There were 
interviews with 59 of the original respondents. For the most part, interviews were 
conducted individually; on two occasions, teachers from the same programme chose to 
be interviewed together. Further information concerning the interview methods is 
contained in Appendix 2. 

A slightly different version of Part 2 of this research is being published as a stand-alone 
publication (Maynard and Martinez 2002). For ease of reference, the structure of Part 2 
of the report is similar to that in Part 1 . Some overall conclusions are drawn at the end 
of the report. 



Overall findings 

Teachers interviewed had strong views about which factors most affected retention and 
achievement, although these were sometimes contradictory and required further 
clarification. For example, teams with the highest retention and achievement often 
claimed that negative factors were affecting their courses when, statistically, retention 
and achievement had remained consistently high for several years. Conversely, many 
teachers on courses with low retention and achievement claimed that students left 
because of economic or social circumstances, although courses with high retention and 
achievement existed in their colleges and recruited a similar student cohort. 

Overall, the interviews demonstrated that the original hypotheses are broadly correct: 
teachers in improving courses largely attribute improvements to their own agency and 
teachers in courses with low retention and achievement largely attribute their position 
to factors over which they have little or no control. That said, it also became clear that 
staff on courses with the highest retention and achievement brought to their courses 
some specific attitudes, aptitudes and strategies that contributed significantly to their 
success. These seem to contribute to what we might term effective teaching. 

Conversely, staff in courses with the lowest retention and achievement appear to have 
brought to their courses specific attitudes and behaviours that contributed significantly 
to a failure to retain students or to improve success rates. This does not necessarily 
mean that staff on lower baseline courses believed themselves to be any the less 
effective as teachers, since they rarely associated relatively poor retention and 
achievement with their teaching. 

The case is different again when courses with previously high retention and 
achievement go into decline. This was associated with a loss of confidence among 
teachers in their ability to manage their courses effectively. Indeed, we have already 
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seen from the survey that teachers on such courses gave the least optimistic 
assessments of the impact of different changes on the success of their students. 

When courses with low retention and achievement improved, it was usually because 
the teachers began to adopt strategies and processes they believed to be responsible 
for contributing to improved retention and achievement on other courses in their 
department or college. This sometimes occurred because of a change in team 
leadership or college senior management, but more often came about because the 
team accepted that a new strategy or process seen to be effective elsewhere could be 
equally effective for them. 

Although effective teachers adopted a number of different strategies that clearly 
affected retention and achievement, it is impossible either to produce a checklist or to 
isolate one process as more effective than another. Nor is there much value in trying to 
determine which positive factors outweighed which negative ones. There is some 
evidence, however, suggesting that effective teachers, both as individuals and team 
members, are strongly on the side of students - regardless of programme level - and 
dedicated to student success. 

The questions in this section asked teachers to rate the impact of curriculum-related 
issues like course content, level and scheme of work on student retention and 
achievement. They were also able to say whether they thought that issues like 
enrichment, tutoring arrangements and key skills were positive or negative factors. 

Teachers on courses with the highest retention and achievement rates saw the 
curriculum as an important element in student retention and achievement and believed 
in the systematic preparation of student-centred schemes of work which are interesting 
and stimulating, but do not involve any ‘dumbing down’. Their view was that teachers 
should not only understand individual student motivation for taking the course but also 
the need to maintain that motivation throughout the programme. Courses with high 
retention and achievement included a strong practical element (for both academic and 
non-academic courses) and addressed literacy/numeracy support needs at an early 
stage. Responses about key skills, however, were ambivalent. Some teachers saw 
them as a valuable initiative, others did not. Generally speaking, teams thought it more 
important to plan enrichment activities that complemented the main course (including 
outside speakers, residentials, etc) and to ensure that all students are tracked and 
supported throughout their programmes . 

Traditionally, when I came here 4 years ago the retention rate for 
Computing was poor, so one of the things I was keen to do was to 
actually address that by offering ICT [information and 
communications technology] as an alternative, because it had a 
strong user focus and was more relevant to student needs. 

Teachers of courses/programmes with a low baseline and declining retention and 
achievement appeared either to be complacent about the curriculum or to assume no 
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changes were necessary. Some of those whose courses were most in decline, 
confessed to having no schemes of work and/or no enrichment. On one course 
students were not given handouts because they were expected to buy their own books, 
even though many chose not to do so. On another course, the students had been 
labelled as ‘not interested in the curriculum anyway’. 

Teachers on courses that had had a high baseline in retention and achievement, but 
were now declining had begun to adopt some of the attitudes displayed by teachers on 
low baseline courses. They accepted that their courses were becoming ‘bog standard’ 
or out of touch with both student needs and the needs of industry, but felt unable to do 
anything about it. This loss of faith in the quality of their own curriculum led to some 
conveying their uncertainty and negativity to their students, while others admitted taking 
on students who were not recruited with integrity. Such students found struggling with 
academic concepts increasingly difficult and teachers felt that they had insufficient time 
to cover topics in the detail that these students required. 

Conversely, teachers on courses where retention and achievement were improving 
were adopting strategies similar to those with already high baselines. Through 
benchmarking and seeking support from external bodies and exam boards, teams had 
begun to plan more effectively, update content and add enrichment and interest. More 
attention was being paid to ensuring that entry qualifications matched the demands of 
the curriculum and that both the curriculum and student progress were reviewed 
systematically. 



Findings for specific change factors 

Context of the course/programme 

Teachers were asked to assess the impact of direct line management and team morale 
on retention and achievement. They were also asked about the emphasis on improving 
teaching and learning and whether experiments with teaching were encouraged. 

Respondents on courses with the highest retention and achievement were those who 
gave their fullest attention to teaching and learning, both as individual teachers and as 
members of a team. More importantly, effective teachers and effective teams proved to 
be relatively autonomous. In one highly successful college, a senior management team 
had fostered this autonomy with a strong commitment to improving teaching and 
learning. In other colleges, however, autonomous teams developed because 
practitioners sought to distance themselves from the aggressive management styles of 
individual senior managers or senior management teams perceived as being dictatorial 
and too far removed from what one teacher called, ‘the real business of teaching and 
learning’. In one college a successful team had operated for a year without a section 
leader or head of department and was now running courses with high retention and 
achievement in an annexe some distance from the main campus and rarely visited by 
senior management. 
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All respondents saw teamwork as crucial to both effective teaching and student 
success. Effective teams consisted of experienced, qualified practitioners who were 
interested in student success and whose team meetings focused on comprehensive, 
detailed planning and delivery, sharing good practice in teaching and learning, and 
monitoring individual student progress. Effective teams had high morale, met regularly 
and resolved any differences between them at team meetings. They believed that 
students always knew if they were being taught by a happy and stable team and 
responded accordingly. They also believed a good team set sound ground rules for 
students and always tried to motivate and inspire them: 



On low baseline courses, however, some respondents were either complacent about 
the course team or said that the team had grown stale or lost key members. Some 
teams did not meet or felt under pressure when they did so because they were 
constantly asked to respond to new management-led developments, not always 
connected with classroom practice. Several respondents admitted to being in a team 
whose members not only felt negative about all aspects of college work but conveyed 
these attitudes to students in class. As a member of a team with high retention and 
achievement put it: 



Students are very clever; they soon find out if a teacher’s 
bothered about them and - I probably shouldn’t say it - but 
some teachers don’t really care if their students pass or fail 
and don’t feel accountable. Some teachers take the attitude, 
‘Oh no! I don’t teach below advanced level, that’s beneath 



There appears to be a specific pattern to the decline in retention and achievement in 
courses/programmes which had started from a high baseline in 1996/7. Usually, one or 
more incident led to a lowering of individual or team morale. This might be triggered by 
increased workloads, team members becoming ill or absent or a general perception 
that line managers and senior management teams were paying less attention to 
teaching, learning and staff development. In one instance, a college manager refused 
to allow an annual end-of-year student art exhibition to continue on the grounds of 
‘health and safety’, leaving the teacher involved feeling bitter and unappreciated. 

By contrast, teachers whose courses began to improve retention and achievement from 
a low baseline started to demonstrate greater team autonomy. They became a more 
united, more close=knit group, developed a team philosophy and placed greater 
emphasis on teaching and learning. One team was enabled to do this through a new 
senior manager who supported staff and wanted to improve quality, but other teams 
developed a more autonomous way of working through their own initiative. Another 



The team has been solid for several years. We all know what 
we’re doing and the students have got the best tutor they 
could have for each particular part of their course. 



me. 
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team started to introduce value added and absence reporting systems as a means of 
improving retention and achievement. 



Providing effective assessment and feedback 

Effective teachers who commented in detail about student assessment saw students as 
the focus of their work. There was a general view that students, ‘need to feel they’re 
achieving. If you work for them, they’ll work for you’. Team members, therefore, saw 
assessment as a key part of improving student achievement, often emphasising that it 
was a two-way process in which the student has as much input as the teacher. 

The team made things more manageable for the students so 
that it was clearer to them how much assessment they would 
face and when. Because they could see the structure better 
they were more encouraged. 

While it may be helpful for an institution or team to have an assessment policy, 
assessment handbooks and log books, effective teachers placed more emphasis on 
students having a clear understanding of how they are assessed and the criteria they 
are being assessed against. There was a strong view expressed by several 
respondents that assessment should be scheduled and manageable with a particular 
emphasis on assessment early in the course, so that students can understand the 
assessment criteria and apply their understanding to improve their work. It was also felt 
important that students should experience early success and be given both detailed, 
constructive feedback and individual support to improve. When students had 
confidence in their ability to achieve, they could then be encouraged to do more of their 
own research and adopt a more reflective approach to their learning. 

It was interesting to note that courses with the highest retention and achievement had 
teachers who were sensitive to the demands assessment makes on students and the 
need to make assessment tasks more comprehensible and achievable, but without 
losing any rigour. One team ensured that on a low-level practical course, assessment 
was as skill-based and as non-threatening as possible. Another team used innovative 
course-builder software to set up assessments on the college’s intranet. One 
respondent also pointed out that teachers need to be aware that college assessment 
schedules and processes: 

...do not always take account of the fact that a student’s 
personal problems can, on occasions, interfere with their work 
and that it is no good putting them under pressure to achieve 
without resolving the problems. Pressure to get students to 
complete assignments when they are experiencing major 
personal problems is [more] liable to facilitate drop-out than 
aid retention. 
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Teachers on courses with low baselines for retention and achievement saw 
assessment and feedback as one of their least important priorities. They had either no 
view on the matter or felt the team knew what they were doing so there was not a 
problem. Two respondents, however, blamed the students, either for not understanding 
their assessment requirements in the first place or failing to complete assessments, 
‘because they [adult students] were trying to juggle too many commitments’. One 
teacher suggested that the reasons some students didn’t hand work in was that they 
didn’t want to achieve at that level, while another said that he always expected 25% of 
his students to drop out anyway. 

Teachers where high retention and achievement were declining attributed their 
problems to a mixture of factors both within and beyond their control. There was a view 
that staff did not always understand the assessment process themselves or that too 
much reliance was placed on giving students an assessment handbook and assuming 
that they would read it. One respondent felt that students had too much assessment, 
with too many assignments being required at the same time; another thought that 
assessment was becoming less rigorous and that, even then, the students were not up 
to the required assessment level. In one instance, staff imposed rigid deadlines and 
refused to accept student work after the required date. There was also a general 
complaint that students no longer did homework because they were too busy earning 
money in part-time employment. 

On low baseline courses where retention and achievement were improving, teachers 
were beginning to adopt similar practices to teachers on high baseline courses. Care 
was being taken that students now understood the assessment process and evidence 
requirements, and efforts were being made, especially on lower level courses, to 
motivate students to achieve. One teacher was rewriting assignments to make them 
more interesting, humorous and relevant while another was introducing more support 
for basic skills. 

Staffing 

This part of the interviews asked for responses that would clarify both the role of the 
team and the mix of skills within the team or programme area. It also provided a 
second opportunity for respondents to clarify their views on departmental/faculty 
leadership and the role of senior curriculum leadership within the college. The interview 
outcomes were interesting in that the highest achieving teams emphasised again the 
importance of effective and autonomous teams and had little to say about line 
managers or senior managers. 

Courses with high retention and achievement rates were usually taught by full-time staff 
whose subject/commercial knowledge was matched by their ability to teach. The 
course team was regarded as crucial, with particular emphasis placed on the need for 
individuals to have complementary skills and, in the case of vocational programmes, a 
strong industrial background. It is interesting to note that while a stable team enables 
detailed and comprehensive curriculum planning and updating to take place, effective 
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teams can also cope with staff turnover and increased use of part-time staff. This, 
again, appears to be linked with team autonomy since commitment to the team was 
seen as more important than commitment to the section manager or departmental 
head. What most teams had in common, however, was ‘a commitment to do the best 
for students’. 

Teachers on courses with low retention and achievement were the most aggrieved 
about their status and working conditions. They perceived they were doing all that they 
could in circumstances that were largely beyond their control. They stated that some 
problems were due to poor cover arrangements and others occurred because their 
colleges now employed too many part-time staff who knew very little about the 
education system or the students they were teaching. In one instance, teachers were 
so dissatisfied by senior management proposals for redundancies and restructuring 
that they conveyed their views to the students with the effect that all of them left the 
programme! 

A similar malaise was found among respondents whose previously high retention and 
achievement rates were now in decline. These teachers perceived senior managers to 
be deliberately reducing team numbers and relying more and more on part-time staff to 
cover the gaps. This left specialist areas covered inadequately or forced the team to 
‘cobble together missing elements of expertise’. In one instance, a teacher’s 
enthusiasm began to wane because of a feeling that, ‘all new ideas are scotched’. 
Others attributed declining retention and achievement to senior managers’ inability to 
respond positively to the need to provide cover when staff were absent or ill. 

Staff on courses where low retention and achievement were improving did not attribute 
these improvements to any significant changes in academic or vocational staffing 
levels. They did, however, emphasise a change to more positive approaches by senior 
managers, and staff development that valued the teacher and sought to improve 
strategies for teaching and learning. 

Quality systems 

Questions on quality asked teachers to evaluate the impact of changes in a number of 
quality factors such as self-assessment, MIS and the observations of teaching and 
learning. They were also asked to indicate what value they placed on student feedback 
and the way managers monitored course performance. 

As might perhaps be expected, this proved to be one of the most contentious areas, 
and teacher attitudes were divided. Some quality systems were perceived as directly 
relating to teaching and learning, but others were seen as directly controlled by senior 
management and not related to classroom activity eg MIS, management-led target- 
setting, and so on. Teachers on courses with high retention and achievement, believed 
that the crucial element of quality improvement was to act on information received, 
whether it was from MIS, student feedback, the observation of teaching and learning, 
internal or external verifiers or self-assessment. Where systems were management led 
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and focused on MIS, however, staff views often echoed those of one respondent who 
said, ‘You’re just chasing paperwork which then goes into a black hole’. 

The general view was that, ’The teaching observation system with positive feedback 
has far more impact than MIS’ and ‘Direct contact with students tells you whether they 
are content or not’. Some respondents, however, had particularly positive views about 
observations. 



More observation of each other’s teaching leads to sharing of 
good practice and we certainly increased the observation of 
each other’s teaching - peer observation as well as 
observation for the sake of appraisal. 

As well as using the college’s general questionnaire we 
increased the amount of feedback we were getting from 
students by using our own subject questionnaire and used the 
outcomes to feed into our action planning and scheme of work 

planning. 

Teachers on courses with low or declining retention and achievement seemed either to 
be complacent about the quality of their provision or to adopt the attitude that quality 
was already embedded in the team and ‘we know what’s best’. MIS was generally 
regarded as ‘useless’ and one respondent thought that student retention was better 
before there were any quality systems in place that suggested otherwise. Those on 
courses with declining retention and achievement were the most critical. One teacher 
stated, rather acerbically, ‘Quality is a euphemism for paperwork, isn’t it?’ Another 
blamed college management for having quality systems in place but never acting on 
student feedback. This, it was claimed, lowered student morale on the course in 
question and led to a downward spiral in retention and achievement. 

Where previously high retention and achievement rates were declining, teachers were 
divided between the view that ‘Quality is fine’ and the view that ‘Quality is non-existent’. 
No interviewees made any positive or negative comments about actual quality systems, 
although there was a complaint that, ‘Strategies seem to be used as a process rather 
than getting results’. 

There were some positive responses about quality from teachers on courses where low 
retention and achievement rates are improving. Yet while there was praise for systems 
in place for the observation of teaching and learning and even the Inspection process, 
there were still very negative views about management-controlled quality systems. 

The quality system is a joke - written for academics - and far 
more importance is attached to it than there ought to be. 
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The mixed responses to quality systems and, in particular, the negative views of what 
were perceived as management-led systems, would suggest some of the other types of 
change (notably teaching, curriculum and assessment) are seen by teachers as having 
a greater positive impact on student success. 

Resources 

When interviews focused on this area, teachers were asked to clarify the positive or 
negative impact of changes to preparation time, the number of teaching hours they had 
and the availability of essential equipment and technical support. They also had the 
opportunity to discuss the accommodation provided for their course or programme. 

In reply, interviewees made a clear distinction between human resources and other 
resources they thought necessary for students to complete their programmes 
successfully. A significant number of staff on courses with high retention and 
achievement rates were dissatisfied with both types of resource and claimed these 
were having a negative impact on retention and achievement. There was 
dissatisfaction with the number of teaching hours, remission (regarded by one 
respondent as ‘a joke’) and the poor quality of many learning environments. 
Interestingly, one course was given a Grade 1 at Inspection, although it was judged to 
be run in poor accommodation. Several teachers cited out-of-date vocational resources 
with one respondent complaining that essential day-to-day resources were never 
ordered on time. 

There was, however, clearly a division between the ‘haves’ and the ‘have nots’. Staff on 
some courses with high retention and achievement rates had equipment of current 
commercial and industrial standard, housed in modern, state-of-the-art 
accommodation. Some colleges had well-advanced intranets and new buildings where 
classrooms were equipped with electronic whiteboards. This division is interesting. 
Although the lack of resources did not seem to be affecting retention and achievement, 
staff clearly felt that it did, and may have been making compensatory efforts in other 
areas of activity to make up for the ‘shortfall’. 

It seems clear that students do convey their sense of dissatisfaction to staff when 
learning environments are below par and there are insufficient and often outdated 
resources and equipment. 

This concern becomes more apparent on courses with low retention and achievement. 
Here, perhaps because there are no compensatory factors, poor resourcing might 
simply add to students’ negative experiences. One teacher cited instances of students 
able to word-process work but unable to print it off because printers had broken down. 
Another claimed his workshop was now 40 years out of date and no longer relevant to 
the syllabus. The most negative comments suggested that human resources were 
almost as overused as some of the workshop equipment and staff had no time to 
prepare lessons, insufficient delivery time and too much college paperwork. 
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Courses where high rates of retention and achievement were in decline had the most 
negative responses of all. Although one or two thought this was not a key issue 
(because teaching methods were far more significant) there was growing frustration 
that mismanagement of human (and other) resources made teachers look incompetent 
in the classroom. One teacher complained that, The OHPs don’t work, the computers 
frequently crash and that makes the students lose confidence’. Another commented 
that a lack of preparation time and insufficient full-time staff ‘...means teachers come 
across as harassed, lacking preparation and stressed’. 

Teachers on courses where previously low retention and achievement were improving 
reflected the same division of opinion and experience as those whose retention and 
achievement rates were high. On the one hand, there were those who believed new 
resources were helping to raise retention and achievement. On the other hand, 
retention and achievement were also rising on courses where, as one respondent put 
it: There is out-of-date equipment, lack of computers, poor classrooms and too much 
use of acetates and boardwork’. On another course, retention and achievement were 
improving, although the teacher reported that the equipment was, ‘poor and frequently 
broken’ and that there was ‘a lack of access to computers and software’. 

Student cohort 

Teachers were asked to rate the impact of the changes to the student cohort in terms 
of ability and qualifications at the start of the course/programme. They were also asked 
to determine the impact of students leaving for employment, the amount of part-time 
work they did and the impact of financial or personal problems on their studies. 

This aspect of the survey attracted by far the most comment at interview. Teacher 
perceptions, however, proved difficult to interpret and understand. On courses with high 
retention and achievement that were still improving, the general view was that while 
adult student motivation remained high, the 16-19-year-old student cohort was less 
committed to learning than in previous years. Teachers felt that the main reasons for 
this lack of commitment lay in more students having part-time jobs and more personal 
problems interfering with their studies. Students were felt to be ‘not as systematic and 
hard-working.’ One respondent felt: 

Some students’ part-time work commitments are more 
important to them than their commitment to college - they 
don’t attend as well or achieve as highly as they could. 

Teachers’ views were more mixed on courses that were maintaining a high level of 
retention and achievement. While there was a broad view that current 16-19-year-old 
students were not as academic as they used to be and that part- time employment was 
an issue, many felt that teachers could and should cope with these realities, by 
employing a range of strategies. One respondent suggested, for example, that students 
were not always sufficiently informed about college courses, the amount of work and 
the distinctions between course levels. This meant that some students believed that all 
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vocational courses were 100% practical and that there was little or no written work 
involved. Appropriate strategies would therefore focus on information, advice, guidance 
and induction. Another teacher felt that tutors now needed to know more about the 
impact of students’ personal problems on their ability to study and needed to develop 
counselling skills to deal with these issues. 

Most teachers on courses with high retention and achievement rates believed that it is 
important to understand students as individuals and motivate them to succeed. This 
was also seen as an ongoing process, necessary to keep students on track. Where 
appropriate, literacy and numeracy support was seen as an important part of that 
process. One teacher also stressed the importance of encouraging students to succeed 
beyond their own expectations; she also encouraged individual aspirations and 
personal ambitions. Another stressed the importance of motivating students to take on 
individual responsibility. This view is exemplified in the following statements from 
teachers: 



Modern students have low attention spans and are not as 
academic as they were. This has to be compensated for by 
better understanding of principles of learning and students 
need to be more open with staff about their problems and their 
part-time work. 

The student cohort has changed significantly, but I’m not 
blaming the students. We need to respond to them and not the 
other way round. General cultural attitudes of young people 
are changing anyway. 

These last two quotes may help to explain why retention and achievement remained 
high on courses with a high baseline position. There remains the paradox, however, 
that teachers on courses which started with high retention and achievement rates, and 
which improved still further, were generally rather critical of changes in the attitude and 
capability of their younger students. When pressed at interview, teachers on courses 
which were still improving their retention and achievement agreed that while their 
students were still staying on course and achieving, they were thought not to be the 
‘high flyers’ of yester-year, nor were they thought to be getting as many grade As or 
Distinctions. Neither of these aspects of the student cohort was systematically tested in 
the research so it is not possible to evaluate them in detail. 

Teachers on courses with high levels of retention and achievement occasionally 
expressed a view that the Educational Maintenance Award (EMAs) did help some 
students with financial problems to stay on course and succeed. Not surprisingly, this 
view was most prevalent in courses that recruited from areas of high rural or urban 
deprivation. 

Teachers on courses where retention and achievement were low or falling further, 
however, saw EMAs as making a bad student cohort worse. EMAs simply brought in 
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students who ‘just want the money’ and ‘a disruptive element that drives better students 
away’. Teachers on these low baseline courses also saw their student intakes as ‘dire’ 
and ‘of low ability’. In one instance, an interviewee chose to link EMAs and ability with 
race, suggesting that ‘we’re seen as an Asian college now’ and ‘two-thirds of our 
problem is Asian girls’. 

These teachers did not, therefore, express the same kind of view as teachers on 
courses where high retention and achievement were still improving. Instead, many 
teachers whose courses had low and declining retention and achievement appeared to 
be willing to label students, and stereotype them in terms of race, gender and class. 
Nor did these teachers appear to be able to identify possible solutions. They expressed 
views that ‘it takes too long to determine ‘at-risk’ students’, or that students on the 
courses had ‘a general culture of not succeeding... .not being on time, not handing work 
in on time - lack ability to think about their futures’. 

Teachers on courses with low achievement and success rates believed that college 
management teams deliberately recruited these students to get funding and, as a 
consequence, did not recruit with integrity. This meant that students were on courses at 
an inappropriate level, which demotivated them and led them to demotivate others. 
Students, in their view, were under pressure because they were unable to complete 
work and had problems at home anyway. 

Teachers whose high rates of retention and achievement were in decline expressed 
almost exactly the same attitudes. They also blamed college managers for making 
them recruit weak students who would leave and also claimed that because ‘support for 
weak students is finite’, too many ‘at-risk’ students were identified too late. Students 
were also thought to take on part-time jobs ‘to fund a student lifestyle’ and too much 
part-time work made them too tired to study at college - not that they had any study 
skills anyway! One teacher managed to link most of these issues together and claimed 
that 25% of students on the course should not be there in any case, so it was hardly 
surprising that they were not retained and did not achieve. Another summarised most 
of the views expressed by saying that the course recruited: 

...weak non-achievers... who have personal problems. They 
are immature and not prepared to study in a serious way. 

They have been turned away from better colleges and taken 
on here. They are accepted even though it is known they will 
not be retained. 

Teachers on courses where retention and achievement were improving expressed 
some of these attitudes, but had begun to analyse their student cohorts and take action 
to improve matters. One accepted that there was a problem with part-time jobs, but had 
also discovered one student who had to work at a fast food outlet to support his 
mother. Another believed that part-time work that complemented a vocational course 
ought to be encouraged and that the EMA, far from being a negative factor, was 
actually a godsend in what was a very deprived area. Several teachers also said that in 
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their colleges students were now being recruited with integrity and that they were now 
taking steps to develop student motivation and correct any misconceptions about the 
course, early on. 

Teachers on these courses also believed that staff-student relationships and 
relationships between peers were of particular importance. Care was taken to develop 
a good relationship with students and not to allow negative peer pressure to affect 
retention and achievement. In one instance, strong students were encouraged to 
support weaker ones as ‘buddies’. Another course team went out of its way to have an 
enjoyable induction programme that both laid ground rules and built up a group identity. 
One very committed team leader of a vocational course that had shown rapid 
improvement said of her group: 

They’re not the best motivated of students, but you have to tap 
into their uniqueness to help them become motivated. At the 
moment the students are fasting for Ramadan, so yesterday I 
fasted so I could gain their respect and understand more of 
what they’re going through and how difficult it is. 

Support for students 

Although teachers on courses where high retention and achievement were improving 
appeared to have a pessimistic view of changes to their student cohorts, this was not 
reflected in what they did in practice to support students. Instead, there was a desire to 
provide students with any support within their means and to refer them on to specialists 
when it was more appropriate to do so. This was done through early identification of ‘at- 
risk’ students and referrals to Student Services at an early stage. On one course it was 
felt that personal tutoring had prevented quite a few students from leaving and parental 
involvement was sought and encouraged. On two other programmes, staff put on 
additional support workshops in their own time and in one of these all students had to 
attend at least one lunchtime drop-in workshop and report back in class on what they 
did there. One of these courses also ensured that: 

Additional workshops were put on in exam techniques and 
requirements for the students who needed them and I think it 
was beneficial for them. It helped them to improve their 

performance. 

On courses where high retention and achievement rates were maintained, staff had the 
same strategies. The staff in most teams were pro-active about providing support, 
organising additional support and referring students on to Student Services. One 
interviewee said: 



We take students seriously and give them time and effort. 
Students appreciate help with their assignments and all 
students go to additional support. 
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Another said that she was always in her part-time class 15 minutes before the lesson 
began and stayed there for 15 minutes afterwards. This enabled her to provide extra 
support for students who knew she would be there. She also felt it was crucial to help 
these students to support each other and for them to know what Student Support 
facilities were available in the college and of their right to access them. Most teachers 
interviewed also thought it important to act quickly over student absenteeism or give 
prompt support to any student whose lack of progress was a cause for concern. 
Emphasis was also placed on initial assessment/diagnostic testing and tutorial systems 
that tracked student progress using one-to-one reviews. 

Although one or two staff on courses with low retention and achievement claimed to 
offer personal support to students, most saw gaps in the system beyond their control. 
There was a general feeling that individual ‘key staff were solely responsible for this 
area and things generally went wrong if and when such staff left. Learning support was 
also felt by one respondent to be ‘hit and miss’ and that it took the college a long time 
to work out who was ‘at risk’ and ‘filter them out’ of a programme. One or two teachers 
thought the onus was on students to seek help if they needed it - it was their 
responsibility and nobody else’s. 

Support for teachers 

At interview, teachers whose courses had the highest retention and achievement 
reinforced the view that the most effective teams were autonomous. Few teams, 
however, had actually been empowered by senior managers to be so. Most had taken 
autonomy on themselves because they did not feel supported by senior managers, line 
managers or even clerical workers. One team actually operated successfully for a year 
without a direct line manager or a head of department. A teacher in another team felt 
he had been ‘left to his own devices’ and with no technician in the vocational 
workshops he also had to take on this role as well. Generally speaking, the most 
effective teams set themselves high standards that they refused to lower in the face of 
adversity. Disenchantment with management implied that they were driven more by 
their view of general professional standards of teaching rather than by those standards 
developed within the college. It is interesting to note that most interviewees’ responses 
were focused on their team and what the team did to support students and each other 
rather than on the leadership of more remote college managers. This was 
accompanied by some tart comments about the absence of rewards for their efforts. 
One teacher whose high retention and achievement was still improving commented 
that there were: 



...never any positives, but management quickly pick up on 
negatives - this can lead to student disgruntlement, but in this 
instance it lead to staff-student collusion against college 
management. 
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There were positive comments about staff development, which was felt to boost staff 
morale and assist retention and achievement. Some respondents, however, felt staff 
development sometimes focused on what most concerned managers rather than what 
really supported teachers. There was also much comment that clerical support, where it 
existed, freed teachers from major administrative burdens and allowed them to 
concentrate more on course preparation or being able to put in extra time helping 
students in difficulty. 

Where retention and achievement were poor, teachers seemed to be much more self- 
oriented. They felt most supported by a team of like-minded people and felt that 
management-led support systems often existed in name only. Some teachers felt that 
everything was all right and always had been; others felt this issue had no impact on 
retention and achievement anyway. Some criticisms were also voiced. In one college, 
for example, the respondent said that poor administrative and clerical support meant 
that students often did not receive a reply to their enquiries or were sent inappropriate 
information. In the same college, staff development was said to be reserved for the 
favoured few. 

Teachers on courses where previously high retention and achievement rates were now 
declining were mostly neutral in their views, but several individuals felt that a lack of 
support had made teaching, ‘an onerous, burdensome workload’. Where a single 
member of staff found herself left in sole charge of courses, without any back up, she 
said that she felt, ‘lonely and pressurised’ and that she had had ‘no reward for getting 
results anyway’. 

Teachers on courses where low retention and achievement were improving were also 
mostly neutral, but some felt that there were big improvements taking place in staff 
development and one had a new head of department who was said to be excellent. In 
this college, because all the heads of department were showing a revived interest in 
teaching and learning, it was thought that this was now putting pressure on senior 
managers and they might change too. Another interviewee expressed a similar view 
but in more detail: 

Immediate managers have an understanding of teaching and 
learning and staff needs. Senior managers adopt a top-down 
approach and usually set up teams to solve problems without 
asking the people who actually know. We get little support 
from senior management, only edicts from above. 

Although most teachers gave relatively little emphasis to this area, it became clear that 
the absence of support was a significant factor for those teams that lacked the strength 
and/or the support to operate autonomously. With little or no clerical support and under 
constant pressure to complete growing volumes of management-related paperwork, 
some teachers felt increasingly harassed and under pressure. This, in turn, appears to 
be communicated in the classroom, thus increasing student anxieties. There was little 
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evidence, however, of constructive interventions being made to change this situation by 
college managers or departmental heads. 

Teaching 

Staff on courses with high retention and achievement rates thought that there was a 
particular problem about students’ entry qualifications; as they felt under more pressure 
to take on students with limited qualifications. These teams resisted this pressure, 
however, and continued to select students carefully. In one college, students were not 
only expected to have strong qualifications on entry, but to focus on future career 
prospects and whether the course they had in mind would enable them to progress in 
the interview. Most respondents also thought that induction was crucial to retention and 
achievement and that the programme had to involve active learning and, ideally, should 
be fun. As one teacher put it, her students ‘have got to have a feel-good factor by the 
end of the week’. 

Surprisingly, most teachers in teams with high retention and achievement, did not rate 
teaching methods as a significant factor in retention and achievement, although one 
teacher emphasised that what was taught in the classroom should always be relevant 
and use students’ own experiences. There was, however, a general feeling that 
monitoring student progress was crucial and, where necessary, would involve seeing 
students on a one-to-one basis. 

Teachers on courses with the lowest retention and achievement mostly appeared 
unconcerned about student qualifications on entry. One thought that entry criteria for 
his course were a ‘grey area’; another said that the college only recruited with ‘bums on 
seats’ in mind and that many of these students would not be retained. In his opinion, 
what Marketing needed to do was to find teachers the ‘right’ students. One teacher 
whose course had low retention and achievement that was still falling, believed that the 
role of the teacher and what constituted a good lesson had never changed. What had 
changed was ‘the pressure for data and justifying data. So that makes one take on 
weak students and get rid of them as soon as possible’. This view was shared by 
another respondent who said that too much time was taken up by teachers working on 
college procedures and that, as a team leader, he only had 3 hours remission to deal 
with 300 students. He felt that his skills had been so adversely affected that he 
concluded, ‘I wouldn’t like to be taught by myself. 

Teachers whose classes had the lowest level of retention and achievement also had 
the most pessimistic view of the impact of changes in teaching. In one instance, a 
teacher who was new to the profession, had only encountered negative colleagues and 
could see them convey this negativity to students. This teacher felt that there was 
limited support available to help improve things and obtained the impression that 
regardless of what your aspirations are, ‘You’re just expected to get on with the job’. 

Courses where once high retention and achievement were declining had teachers 
whose responses were mostly neutral and who felt that changes to teaching had no 



impact. There was some evidence, however, that students on one programme were no 
longer being recruited with integrity and teachers identified a need to review 
admissions and induction procedures. Several interviewees felt there was no longer 
enough time to monitor student progress in class or to give individuals the attention 
they deserved. 

The most positive responses, however, came from teachers on courses with low 
retention and achievement that were improving. The majority of these had resisted 
pressure for ‘bums on seats’ and had become much more proactive in dealing with 
students’ initial enquiries themselves and ‘selling’ the course enthusiastically. Teams 
also appear to have reviewed recruitment, induction and teaching and learning. One 
team did not enrol students until the end of week 2 so that students had sufficient 
experience of the programme to know that it was exactly what they wanted. Two teams 
had also reviewed and improved teaching and learning so that teaching methods were 
more student-centred. Most teams felt they were more aware of issues affecting 
students’ social lives and their problems at home by collecting more data and making 
use of it. One teacher felt that the whole team was now much more supportive in its 
relationships with students, while another team leader of a foundation programme said: 

I’ve now got a very good teaching team ...who want to know 
about the students as individuals - they care about them and 
we’re all very positive. Our attitude to student success is ‘yes 
we can do it and yes, we will do it!’. 

It may seem surprising that staff on low baseline, improving courses seem to have 
more positive responses to this aspect of the interviews than staff on courses with high 
retention and achievement. This may be because lecturers on low baseline courses are 
discovering their autonomy as teachers and as teams and are more conscious of their 
power to make a difference, whereas teachers with high retention and achievement 
appear sometimes to take those skills for granted. These differences however, become 
less apparent when comparing interview responses on personal tutoring. 

Personal tutoring 

Teachers on courses with the highest retention and achievement saw recent changes 
to tutorial systems as one of the most important means of aiding retention and 
achievement. This is because one-to-one tutorials, in particular, provided a forum in 
which students can review progress, give their own feedback and discuss any barriers 
to learning. Most tutoring systems involve three review and action-planning sessions 
per year although two colleges have open access systems with tutoring ‘on demand’ 
and where those who need it most receive more support. 

Teachers on high-performing courses saw tutorials as providing students with help and 
direction to match their particular needs. They believe that tutorials show students that 
they are valued as individuals and that this is also a forum for praising student 
achievements and resolving individual pressures. 
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Comments like ‘We have a brilliant system’, ‘We’re doing even more to improve things’ 
and ‘It’s the most important factor in retention and achievement’ were very noticeable 
among this group. One respondent said: 

All our students are closely monitored through tutorials and 
through their individual tutor. If you see that they’re having 
trouble you can help them, hopefully, sort it out, whether it be 
with their coursework or their personal problems ... I think 
we’re good at picking up students who won’t come forward for 
help. You can see the signs when they go quiet or they’re 
walking around with a sour face. 

While one or two teachers on courses with low retention and achievement were 
positive about tutorials, most comments were similar to those that follow: 

You can’t alter bad students, they’re more influenced by peer 

pressure. 

If they’re happy they stay, if they’re not they go. Last year we 
let too many waifs and strays in. 

On several courses, especially those where retention and achievement were still falling, 
there was either no tutorial system at all or it was claimed tutorials didn’t work because 
the staff were disaffected. 

Teachers on courses where high retention and achievement were in decline shared 
these negative opinions. Most claimed there was either not enough time to run one-to- 
one tutorials or that senior managers were not prepared to develop newer and more 
effective tutorial systems. 

Teachers on courses where low retention and achievement rates were improving, 
however, were developing their tutorial systems to review student progress on a one- 
to-one basis, and were seeing improvements to retention and achievement as a result. 
One teacher had trained as a counsellor and was now able to give students individual 
support to resolve what were, in many cases, major crises. As one respondent put it, ‘If 
you’re supported and cared for, you’re more likely to work through problems’. 

Course/programme organisation and administration 

This area of the interviews focused on the teachers' effectiveness in working together 
to organise and manage their programmes. Responses were similar to those 
expressed concerning the impact of changes to the context of courses and 
programmes. They reinforced the view that the most successful teams collaborated 
actively and worked to a common framework. They held regular meetings and reviewed 
the progress of ‘at-risk’ students regularly, some on a weekly basis. All teams had 
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formal minutes and often did their own paperwork in the absence of clerical support. 
Teams placed strong emphasis both on curriculum issues and tracking student 
progress through up-to-date files and log books. In one college good practice was 
shared between departments and strong ground rules were set for all students. Staff at 
this college - like most teachers whose courses had high retention and achievement - 
also expressed a strong belief in treating students as individuals because, in their view, 
‘at the end of the day the students are our customers’. 

Teachers whose courses had low retention and achievement rates sometimes retained 
a strong belief in the team and associated student failings with other areas of the 
college. Thus, one respondent thought The team is excellent. The problem is the 
students and their low attention spans.’ During interviews, a number of criticisms of 
team focus or procedures were voiced. One teacher said that the problem was that 
team meetings were about management issues and self-assessment, not about 
teaching and learning. A second claimed that most of his time was taken up with 
college administration and that he had little time for teaching and learning, let alone 
anything else. In other colleges, however, teams had problems holding meetings 
because their members came from different departments. One interviewee said that his 
team meetings were not organised and team leadership was poor. This led to staff 
backbiting about each other in front of their classes. One teacher said that there were 
no systems to monitor student progress in place, so that the students were 
‘freewheeling along’. 

Similar complaints were heard from teachers whose high baselines for retention and 
achievement were in decline. Most respondents said it was difficult to find time to meet 
and there was no time to focus on the learners and their progress. One admitted that 
the team’s administration was unprofessional and that little monitoring of student 
progress took place. Again, teachers tended to blame what they perceived as an 
emphasis by senior managers on paperwork and form filling rather than the need to 
have personal communication about the students: 

It’s important to monitor students carefully to ensure their 
progress is under review and so that we can ensure their 
results are in line with their target grades ... but when I filled 
this questionnaire in I was of the strong opinion that I wasn’t 
able to administer that task in a professional way because of 
the amount of time needed to do that kind of exercise. 

Where courses with low retention and achievement were improving, the responses 
tended to mirror those where high retention and achievement were the norm. Although 
one team member claimed that team meetings were mostly unstructured and about 
‘fire-fighting’, most interviewees claimed that the introduction of structured, timetabled 
team meetings that shared information about the students had led to improvements in 
retention and achievement. This seemed particularly effective when teams identified 
weaker students at an early stage and began tracking their progress. One team had 
also introduced course reviews that included student representatives. 
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Timetabling 



The last section of the interviews asked teachers to assess the significance of course 
hours and periods timetabled for teaching and private study in terms of impact on 
retention and achievement. Staff on courses with high retention and achievement were 
largely neutral, but did comment that the delivery timetable was now tight, particularly 
as students appeared to place less importance on self-study. One teacher expressed 
the view that while the most able students were better at managing their own time, 
others required additional study periods that needed to be staffed and timetabled. Most 
believed there should be no significant gaps in a student’s time at college and that 
catch-up weeks are now essential to ensure ongoing assessment of learning and give 
students time to catch up on assignment work. 

At the other end of the scale, teachers on courses with low retention and achievement 
were divided. Almost half thought it was an issue of little significance, while the rest had 
a general view that students did not use gaps in their day effectively. One teacher said 
that in his college timetabling, ‘did not focus on students and their needs. Catch-up time 
was not allowed because college managers are only interested in the economics of 
teaching hours rather than their function in relation to learning’. 

Where previously high retention and achievement rates were in decline, there was 
some evidence that timetabling was outside the individual teacher’s control and that 
managers were not timetabling effectively. In one college there were many room 
changes at the start of the college year and some lack of retention could be attributed 
to this. In another, managers appeared not to take account of student concentration 
spans and had timetabled some 4-hour teaching sessions. One practical course had 
had its full-time hours reduced to 15 a week, which meant that not only did students 
have insufficient time to complete the practical work, but that the quantity and quality of 
the work were damaged, both affecting achievement. Where timetabling still involved 
gaps between sessions, several instances were given of students not making efficient 
or effective use of free time. Another view expressed about self-directed study was: 

It’s just a cheap way of running education. You throw the onus 
on the student and they sink or swim. It’s a costcutting 
exercise. I think it’s one of the most important things in terms 
of retention and achievement. 

Where retention and achievement were improving on low baseline courses, teachers 
did not attribute the improvement to timetabling. Most had no comment to make and 
the few who did reiterated the problem of students having too many gaps between 
lessons or teaching hours being crammed because of limited teaching time was 
allocated to a full-time course. 
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Overall perceptions of factors most affecting retention and 
achievement 

In the original questionnaire the final question listed 13 types of factors, and teachers 
were asked to select three changes which had a largely negative impact on retention 
and achievement and three changes which had a largely positive effect. The interviews 
sought to explore this issue further and asked teachers to identify one positive and one 
negative factor that they thought had most impact on retention and achievement. 

As might be expected from the discussion so far, teachers who taught on teams with 
the highest retention and achievement focused on key elements of teaching and 
learning as the most important factors associated with high retention and achievement. 
There was a belief that students must be on the right course in the first place and 
taught by well-trained, well-qualified staff teaching well-prepared lessons. This meant 
recruiting students with integrity, a curriculum delivered in an interesting way and 
programmes enriched beyond the minimum. Teachers felt that there should be 
excellent communication between staff and students, and students should receive lots 
of attention and care. Tutorials were thought to be essential in tracking student 
achievement. Students should experience early achievement or accreditation to help 
retain motivation as the course got underway. Not surprisingly, teachers on these 
courses saw themselves as the keystones of the whole process: 

Staff dedication overcomes problems with resources, lack of 
admin and lack of support. If staff stood back, retention and 
achievement would collapse. 

Most felt that the major negative factors affecting retention and achievement of both 
adult and 16-19 year-olds were their personal problems. A minority felt that the key 
issues were either students being on an inappropriate course or poor college facilities 
and a poor learning environment. 

Teachers on courses with low retention and achievement listed the same positive 
factors, although in rather less detail. Their negative factors, however, reinforced the 
perception gained from other responses, that they considered these outside their 
control. They said that lack of management support, lack of resources and low morale 
led to poor retention and achievement, as did the recruitment of poor quality students. 
One interviewee thought EMAs were a positive factor in retention, another that they 
were a negative factor because they brought in students who were only interested in 
the money. 

Teachers on courses where previously high retention and achievement had declined, 
identified many of the same positive factors. They saw the most positive factors as a 
strong team, good teachers and motivated students receiving strong tutorial support. 
The negative factors were similar to those cited by teams with low retention and 
achievement: lack of preparation time and resources, low morale and an inability, 
because of these, to meet student expectations. 
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Teachers on low baseline courses that were improving their retention and achievement, 
however, were much closer in their responses to teams with the highest retention and 
achievement. Responses here focused on the fact that students should be recruited 
with integrity and that induction should be used to ensure their suitability for the 
programme. It was also thought that teachers should provide an interesting curriculum, 
teach to students’ preferred learning styles and ensure that they had, generally, a good 
experience of college. They should have good staff who were always supportive and 
continuously motivating their students to succeed. 

Teachers on improving courses also identified a mixture of negative features which 
they felt would lead to low retention and achievement. Two of these were staff related 
and suggested that staff with negative attitudes would create the climate for non- 
attendance, as would long lectures without any student interaction. They also cited 
factors beyond the lecturer’s control, such as poor resourcing and lack of staffing. It 
was also thought that retention and achievement would be difficult to improve if 
students were unsuitable for the course in the first place, lacked motivation and were 
poor attenders. 



Drawing some conclusions 

The survey and interviews generated a huge amount of rich, detailed, sometimes 
contradictory but always fascinating, information about teacher perceptions of changes 
in a wide variety of factors, which affected student retention and achievement. 

In very general terms, the research points to both continuities and discontinuities in 
teacher views, and in their conceptualisation of their role and the factors associated 
with the effectiveness of student learning. 

The elements of continuity and agreement within this large and diverse group were 
actually stronger than the researchers had initially supposed. Almost all teachers 
tended to emphasise the importance of good teaching, committed and knowledgeable 
teachers, effective recruitment, induction, monitoring student progress and tutoring, 
within the context of effective and supportive teams. 

This broad consensus extends further, to the identification of factors with a negative 
impact. Resources, particularly pressures on teaching time, because of changes to 
teaching hours and increased administration and paperwork, and poor or outdated 
accommodation or equipment, were seen as having the most negative impact. There 
was a similar consensus concerning what broadly could be considered as management 
failings. Where recruitment policies, timetabling, quality and management information 
systems were not thought to be learner focused, they were usually identified as 
negative factors. Somewhat surprisingly, moreover, there was some agreement across 
the whole group of teachers, that students, particularly younger students, tended to be 
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less highly motivated and have less well-developed study skills and habits of 
successful learning, than had previously been the case. 

There were also some quite pronounced examples of discontinuity, between teachers 
whose programmes had relatively high or low baseline positions, and whose retention 
and achievement rates, over the 3 years, were improving, remaining the same or 
declining. 

This was particularly evident in relation to the views expressed in the survey by 
teachers on courses with high retention rates that improved still further, compared with 
those whose retention and achievement rates had declined from a previously high 
position. Across the whole sample, teachers from improving, high baseline courses, 
tended to be the most optimistic about the positive factors and least pessimistic about 
the negative ones. Conversely, teachers on declining, high baseline courses, tended to 
be least optimistic about the positive factors, including their own teaching, and the most 
pessimistic about the negative. 

Contrary to expectations, in the survey at least, optimism about teaching and related 
positive factors tended to be high among teachers from declining, low baseline 
courses. Equally, and again, contrary to expectations, teachers on improving, low 
baseline courses did not tend to express particularly optimistic views about their 
teaching in the survey. 

This apparent paradox was explored, and to some extent, explained, in the follow-up 
interviews; the discussion of which forms the major part of this report. 

Three further inferences can be drawn from this research: 

• the existence of significantly different ways of conceptualising 
teaching and the role of teachers, which have some major 
implications for our view of effective teaching 

• the importance of largely implicit and unwritten contracts or 
agreements, between teachers, between teachers and students, 
and between teachers and managers 

• the implications of the research for future efforts to improve student 
retention and achievement rates in colleges, and, perhaps, by 
extension in work-based and adult and community education. 



Teachers’ conceptions of teaching 

The survey and the subsequent interviews offered teachers the opportunities to 
express their views on a wide range of issues likely to affect retention and 
achievement. Their responses indicate areas that most concerned them and imply 
some strongly held beliefs about their role as teachers and their attitudes to students. 
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Teachers on courses with high retention and achievement rates seem to conceptualise 
their job as being about teaching and learning and having the autonomy to make 
professional judgements. They would like to share this ethos with college managers, 
but seem able to embody it in their practice in any case, through the operation of more 
or less autonomous course or programme teams. 

As individuals, these teachers also have a strong belief in their own knowledge, skills 
and expertise and in the quality of their teaching. This means that they welcomed the 
introduction of teaching observations and actively sought feedback from their students. 
Not surprisingly, they view the quality of human resources as equally, if not more, 
important as books, equipment and supportive learning environments. 

These concepts have their strongest embodiment in these teachers’ belief in strong, 
self-supporting and stable course teams who are multi-skilled and set themselves high 
standards in all areas of their work. Such teams focus on course content, curriculum 
delivery, supporting students and reviewing their progress. 

Teachers on courses with the highest retention and achievement rates may express 
doubts about their students’ entry qualifications and levels of ability, but they believe in 
recruitment with integrity and actively seek to understand the student cohort that they 
have to work with. Such teachers have a desire to support students and do their best 
for them. They identify ‘at-risk’ students as early as possible and are sensitive to the 
demands the course makes on students as well as pressures they experience in their 
personal lives. They are also aware that today’s young people have different cultural 
values, hence the need to have a timetable that ensures students are as fully occupied 
in learning as possible during the college day. 

Such teachers also genuinely care about student progress and have a strong belief that 
students benefit from new tutoring processes involving target setting and review. They 
do not, however, believe in, for example, value added systems for their own sake. Old- 
fashioned as it may be, the most successful teachers believe that students need 
‘tender loving care’, something not necessarily always provided at home. 

What might be inferred from these statements is an underlying concept of teaching that 
is based on an interaction and partnership between teachers and learners. Effective 
teachers in this context are teachers who adapt and develop their teaching to meet the 
needs of their learners. 

Teachers on courses with low or declining retention and achievement appear to fall into 
two specific categories with regard to their concept of their role. One category of 
teachers believes that nothing about their role has changed or needs to change. They 
have a strong belief that they are already adequately fulfilling their professional role and 
if things need changing, they have little or no scope or autonomy to change them. 

The second category of teachers on courses with low retention and achievement could 
be described as disenchanted, or even disaffected. They feel that their status has been 
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undermined by management-led initiatives which are threatening enough when they 
involve changing professional practice, but overwhelmingly so when such changes 
coincide with restructuring. 

Both categories of teacher tend to turn to the course team for mutual support, largely to 
retain a sense of status and to reinforce the sense that someone else is to blame for 
poor retention and achievement. This blame usually attaches itself to senior managers 
or students, although some team members criticise each other for relatively poor 
performance on their courses. 

Such teachers seem to blame their students the most for poor retention and 
achievement. Teachers, in their view, cannot deliver what senior managers want 
because the students are perceived as not having the necessary ability, are poorly 
motivated or have a culture that is not conducive to learning. These are not the type of 
student such teachers want and they are seen as actually being brought in by college 
management only for the funding units they represent. This may explain why some 
teachers in this category believe that a proportion of such students must be filtered out 
of the college as soon as possible. There is a belief that such students should not really 
be enrolled in the first place, because they don’t want to achieve anyway. 

Teachers whose previously high retention and achievement are now in decline, appear 
to share some of the concepts of teachers whose high retention and achievement is 
being maintained or is still improving. The interviews suggest that something has 
happened to make them disenchanted and to take on attitudes and behaviours 
associated with lower achieving teams. This might be a result of sudden changes within 
the team, perceived changes in management attitudes or a belief that key changes are 
inherently wrong anyway. When this change is coupled with a long-standing belief that 
one’s good work has never been acknowledged or rewarded anyway, this results in a 
sense of low self-esteem that then becomes projected into the workplace. Teachers in 
this category appear to see the learning environment, learning resources and students 
in a particularly negative light. 

These teachers now begin to share some of the same views of students as staff on 
courses with low retention and achievement. They believe that students are not being 
recruited with integrity, but assert this is a senior management responsibility and not 
their own. They also expressed beliefs, more pronounced than in any other category, 
that such students need more rigorous assessment and stricter deadlines and that if 
students experienced problems there was no time to deal with them. 

Teachers on courses where low retention and achievement were improving began to 
change their perceptions of themselves and their teams. This sometimes happened 
because they were encouraged to do so by a new line manager, but usually because 
teachers recognised there were new strategies available that might make a difference 
to student retention and achievement. 
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This change in attitude appears to be associated with a perception that teams can be 
strong and that teachers can make a difference. Teams appear to become more 
structured as a result and to place more emphasis on quality systems, like the 
observation of teaching and learning and student feedback. 

Teachers in this category still had some negative views about their students, but they 
began to analyse their student cohorts and tried to understand student lifestyles. 
Teams in this category begin to recruit with more integrity, focus on student 
performance and understand why they are not achieving. Caring for students and 
helping them are seen to get results. 



Unwritten contracts and agreements 

Over the last 10 years, much work has been undertaken to formalise further education. 
Both staff and students have written contracts that attempt to determine working 
practices in the college and the classroom. The present research suggests, however, 
that there are other, invisible contracts between both teachers and managers and 
teachers and students. These may, in reality, be much stronger than those that, on 
paper, appear more contractual and binding. In the light of this research, some 
tentative conclusions can be drawn concerning the nature of such implicit contracts. 

As far as teachers on courses with high retention and achievement are concerned, the 
most important unwritten contracts are with other teachers in the course team and with 
students. The contract between team members is to be mutually supportive and 
dedicate themselves to student success. The contract with students is to provide 
quality teaching and tutorial support in return for retention and achievement. While an 
unwritten contract with the team, however, might be of long-standing, those with 
students are subject to continuous ongoing review. 

This is not to say that teachers on courses with the highest retention and achievement 
place no importance on unwritten contracts with college managers. They do - but only 
if they perceive that managers are reasonably committed to teaching and learning and 
supportive of both staff development and student success. 

Teachers on courses where retention and achievement rates remain low or decline 
further appear to believe that, as teachers, they have an unwritten contract with college 
managers. That contract appears to rest on the managers’ ability to support them in 
their role and provide a working environment where they can be successful teachers. 
There is some evidence that such teachers do try to conform to management-led 
process improvements while grappling with the everyday work of teaching and learning. 
When management actions subsequently do not respect or support their professional 
role, but focus instead on a teacher’s failure to achieve, the unwritten contract is 
broken. 
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When courses with previously high retention and achievement decline, there is some 
evidence that teachers on these courses have not only lost the professional satisfaction 
associated with student success, but are also losing the part of the unwritten contract 
with managers to the effect that success is not only acknowledged, but might one day 
be rewarded with promotion. It also seems likely that changes within a long-standing 
team have not generated a new unwritten contract with team replacements. That said, 
there are teams that admit their staleness and the need to review the way they operate. 

Teachers on courses where low retention and achievement are improving, appear 
capable of making a range of unwritten contracts. There is evidence that they will 
respond to new managers in return for staff development and an acknowledgement 
that the course is improving. They will also make the same unwritten contracts within 
the course team and with students as those made by teachers on courses with high 
retention and achievement. 



Improving retention and achievement 

This research has focused on teacher perception of a range of change factors likely to 
have an impact on student retention and achievement. Through interviewing staff 
teaching on a range of courses, this project has sought to determine what teachers 
regard as the most significant positive and negative factors accounting for differences 
between courses which have the highest retention and achievement and those where 
retention and achievement is declining. The purpose of the research was not to 
produce a checklist of best practice, but to understand better what ideas and attitudes 
underpin effective teaching and learning. 

The outcomes of the research suggest that all teachers bring to their work specific 
aptitudes, attitudes and strategies which affect student retention and achievement. 
More importantly, these aptitudes, attitudes and strategies are quite similar within each 
category chosen for this study. There are, therefore, some reasonably distinct and 
predominant patterns of thinking and behaviour on courses with high retention and 
achievement, low retention and achievement and on those where high retention and 
achievement is in decline or where low retention and achievement is improving. 

While the research also bears out the initial hypotheses, that teachers who are most 
successful attribute success to their own agency and that those who are not as 
successful attribute this to factors, beyond their control, it would be unwise to assume 
that one cannot intervene in this behaviour. What is crucial is for college managers and 
other stakeholders outside colleges, to realise that improvement efforts that do not 
engage the ways that teachers conceptualise their role and their value systems, are 
likely to be ineffectual. 

For most of the 1990s, successive governments have attached a high priority to 
improving retention and achievement in colleges. This policy imperative was embodied 
in funding and inspection frameworks; colleges that were seen to be under-achieving 
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were ‘named and shamed’. Colleges have been required to set public and published 
improvement targets, agreed by their governing bodies, and quite large sums of 
discretionary funding (in the shape of the Standards Fund) have been allocated to high- 
performing colleges, to disseminate their practice, and to low performing colleges to 
improve theirs. 

Improvements in retention and achievement rates are central to the development plans 
of many colleges, and remain a focus of senior management attention and energy. 
Notwithstanding all this effort, substantial improvements have proved elusive. Retention 
rates have barely improved over the last 4 years for which data is available (1996- 
2000). Achievement rates have improved over the same period, but the rate of 
improvement appears to be slowing down. 

The present research offers some insights into the real difficulties in making sustained 
improvements in student retention and achievement, and suggests some ways forward. 
It suggests that improvement efforts that are not focused almost obsessively on 
students’ learning and teachers’ teaching and the interaction between the two will not 
be successful. This is in line with a large body of contemporary research on school 
improvement. 

The evidence suggests, however, that this apparently straightforward conclusion is 
complicated by three intervening factors: 

• teacher concepts of teaching and attitudes towards their role and 
their students 

• the operation of teaching teams 

• the existence of implicit or unwritten contracts between teachers 
and managers. 

Staff development, teacher training, qualified teacher status and teaching standards all 
have a role in the development of the competences and skills of teachers. Neither they, 
nor any of the other policy levers listed above appear to have been particularly 
successful in influencing the behaviours and views of some of the experienced 
teachers who were interviewed in the course of the present research. 

Concepts of teaching are deeply embedded in teachers’ values and beliefs, and in 
large part, reflect those values (Showers et al. 1987, Borko and Putnam 1995). It is 
notoriously difficult to change values, but it is possible (Wasley 1994, Fullan 1992). To 
do so, however, requires a fundamental and deepseated commitment by managers at 
all levels, to commit to and support such change, and to do so consistently, not least 
through the ways that they provide models through their own behaviour. 

Similar, but different conclusions can be drawn concerning teams. Teams have an 
ambivalent role in this research. Effective, confident and autonomous teams appear to 
sustain and enthuse teachers on the most successful courses. But, they also appear to 
support and confirm the belief that there is little or no scope for improvement among 
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teachers on some of the least successful courses. The implication is that improvement 
strategies that seek to engage teachers will also need to engage the teams and other 
communities within which teaching is planned and delivered. 

These conclusions tend to be reinforced by a further inference: the existence and 
importance of implicit or unwritten contracts, particularly contracts between teachers 
and managers. The research indicates that disengagement and a sense of 
powerlessness is associated with perceptions of a cancellation, or breakdown of such 
contracts between teachers and managers. 

The implication is that to improve retention and achievement, managers will need to 
renew the implicit or symbolic contracts with their teachers, if they want teachers, in 
turn, to renew and develop their own partnerships with their students. 
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Appendix 1: Colleges that participated in the survey 



Barnfield College 
Barnet College 
Bolton College 

Bournemouth and Poole College 
City College, Norwich 
Cornwall College 
Exeter College 

Kensington and Chelsea College 
Loughborough College 
Richard Huish College 
Sandwell College 
Sir George Monoux College 
Solihull Sixth Form College 
South Birmingham College 
Truro College 
West Herts College 
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Appendix 2: Research methods 

The survey 



The objectives of the present research were to identify those factors that are most 
closely associated with improvement, maintenance or decline in student outcomes in 
colleges. To this end, a cross-section of colleges was approached to take part in the 
study. 

The research design that was developed in consultation with the volunteer colleges 
during two initial meetings would: 

• take the course/programme as the unit of analysis 

• consist of a survey in the first instance 

• include a structured sample of courses in each of the participating 
colleges 

• select courses with a minimum of 15 students 

• survey only staff who had a minimum of 2 years’ experience of teaching 
on a relevant course. 

The sampling frame would include roughly equal numbers of improving and declining 
courses and roughly equal numbers of courses with a high or low baseline position. In 
both dimensions, the rate of improvement/baseline position was calculated by 
reference to college rather than national norms. This gave four categories of course: 

• low baseline position and improvement 

• courses with a low baseline position and no improvement or decline 

• courses with a high baseline position and improvement 

• courses with a high baseline position and no improvement or decline. 

The baseline position and degree of improvement were calculated by reference to the 
‘success rate’ of a course. The success rate is the proportion of students who start 
(from the first census date of a year of a course) who achieve the qualification aim. In 
other words, success rates can be calculated by multiplying retention rates by the 
achievement rates. 

The sample of colleges is broadly representative of the college sector in England. 
Fifteen colleges took part in a survey: two sixth form colleges and 13 FE or tertiary 
colleges, one of which has a particularly strong emphasis on adult students. Colleges 
are located in five of the nine regions of England and have a variety of local contexts 
ranging from inner city to urban from suburban to small town and rural. 
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The breakdown of completed questionnaires by categories of course is set out in Table 
1 below. 





1 


2 


3 


4 


5 


6 


Total 


Total% 


Courses in each 
category 


18 


39 


32 


22 


19 


6 


136 


100% 


Number of 
questionnaires 
in each 
category 


24 


66 


57 


34 


35 


15 


231 


100% 


% of total 
questionnaires 


10.4 


28.6 


24.7 


14.7 


15.2 


6.5 


100% 


100% 



Key 

1. High baseline, improving 

2. High baseline, maintaining 

3. Low baseline, improving 

4. Low baseline, maintaining 

5. High baseline, declining 

6. Low baseline, declining 



A detailed questionnaire was developed and piloted by three colleges. The 
administrative procedures for the survey were agreed. Simply put, college coordinators 
identified courses within the sampling frame, distributed questionnaires and collected 
the completed questionnaires. Staff were encouraged to complete their questionnaire 
individually under ‘examination conditions’; in other words, to give an immediate 
response to the questionnaire rather than taking it away to research their records or 
discuss it with other members of their team. The college coordinator collected and 
returned the completed questionnaires to LSDA and also completed a small data sheet 
giving quantitative information concerning each of the courses included in the sample. 
Each course was also assigned to one of the four course categories. 

The categorisation made by the course coordinator has been amended, in a number of 
cases, for one or two reasons. Firstly a consideration of the numerical data suggested 
that the original four categories should be expanded to six: 

• low baseline position, improving 

• low baseline position, maintaining 

• low baseline position, declining 

• high baseline position, improving 

• high baseline position, maintaining 

• high baseline position, declining. 

Second, a small number of courses were re-coded to ensure that there was greater 
conformity between the quantitative data and the course categorisation. 
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The interviews 



The second, qualitative part of the project, on which the substance of the present report 
is based, consisted of detailed interviews of 59 staff who completed questionnaires 
(26% of the overall total) to: 

• investigate more fully the responses to the survey 

• clarify and check some of the tentative conclusions drawn from 
the survey 

• provide the opportunity for teachers to discuss, in a relatively 
open and unstructured way, their understandings of the factors 
affecting retention and achievement. 

The interviews were carried out in 11 of the 15 participating colleges between 
November 2000 and January 2002. These were located in five of the nine regions of 
England and have a variety of local contexts ranging from inner city to urban, from 
suburban to small town and rural. Interviews covered a wide range of programmes from 
Foundation to Level 4. Most of the staff were interviewed individually, but there were 
two interviews involving two or three members of the same curriculum team. 

The interviews were conducted in private and were confidential. Care was taken to 
explain the interview procedure and not to condition any response. Where necessary, 
individual responses were subject to further clarification and all interviewees were 
asked if they would like to comment further on any neutral responses they had made 
on their original questionnaire. Staff were also reminded to focus on their responses to 
changes between 1997-2000 rather than comment on the current situation within their 
individual course, college or programme area. 
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Before each interview, interviewees were given the opportunity to remind themselves 
about the questionnaire and their responses. To focus them about the open-ended 
question at the end, they also had the opportunity to read their responses again and to 
select what they considered the most important positive/negative factor. A few 
interviewees did revise their views. One or two interviewees also amended several 
positive and negative responses, on the grounds that the questionnaire had been given 
to them at a difficult time, eg during college restructuring. 

One or two interviewees expressed anxieties concerning college management and 
sought reassurance that what they said would not go back to senior managers. Two 
interviewees refused to be recorded in one college, and in another, a lecturer asked if 
he could have a photocopy of the interviewer’s notes (which were provided). Another 
team leader of a rapidly improving course began the interview quite defensively, but 
warmed considerably when talking about the students and her team. 

Most of the interviewees were open and forthcoming, but one or two of the best 
teachers adopted what could be described as a persona/mask and the interviewer had 
to recognise this and ‘dig’ beneath the surface. The interviewer was conscious of the 
need to make the interviews as open and transparent as possible, while being aware 
that interviewees and college managers may have hidden agendas. Most interviewees 
constantly asked what the interviewer thought about particular situations in their 
colleges. The interviewer was very conscious of the need to avoid expressing an 
opinion or confirming their opinions. 

The interviews took place between five and six months after the questionnaires were 
completed. Interviews can therefore only record individual perceptions at that point in 
time. If a college was experiencing massive restructuring (as one was) the interviews 
were coloured, to some degree, by staff concerns about redundancy/promotion. 
Curriculum 2000 was being implemented at the same time as the research took place, 
and after the period which was the focus of the research (1997-2000). Because it was 
causing problems; many interviewees wanted to comment on this and one or two 
wanted to change their responses because of their experience of Curriculum 2000. The 
interviewer always shifted the focus away from Curriculum 2000 and back to their 
responses at the time the questionnaire was filled in. 

A few interviewees also wished to produce evidence that things on their courses were 
better than they had been. That was something else the interviewer had to ask them to 
put to one side. 
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Appendix 3: Letter to survey respondents and guidance 



3 April 2001 



To: All staff included in the research on 

factors affecting retention and/or achievement 
1997-2000 



Dear Colleague 

Ref: Research into factors affecting retention and/or achievement 

Thank you for your help in taking part in this research. It is the most comprehensive research to 
date based on the views of teachers and tutors in further education on this crucial subject. 

Why should I bother? 

This research will identify more definitely than ever before the key factors which are affecting 
the success of your students. 

You will receive a copy of the national report based on this research. 

Will my answers be confidential? 

Your answers will be treated in confidence by the Learning and Skills Development Agency, 

(the new name for FEDA). No specific course/programme or college will be identified in any 
way in any publication arising from this project. 

How much time does it take? 

The questionnaire should take between 25 and 35 minutes to complete. It does not involve 
going back to course records. 

What happens next? 

Once you have completed the questionnaire, please return it to your college’s coordinator who 
gave it to you. All the questions will be analysed by the Agency and we will send a report back 
to your coordinator. 

Once the research is written up, we will supply your college with enough copies for you and 
your colleagues who have also taken part in the research. 




ERIC 



84 



Who else is taking part in the research? 

Around 1 ,000 staff teaching on over 200 courses/programmes in 20 colleges are taking part in 
this research. The research covers the period 1997/1998 to 1999/2000. You should have 
taught on the course/programme for at least 2 of these 3 academic years. 

Is there any guidance? 

I have attached some brief guidance on completing the questionnaire to this letter. 

Thank you very much again for taking part in this research. I look forward to receiving your 
completed questionnaire. 

Until then, with best wishes. 



Yours sincerely 

P Martinez (Dr) 
Development Adviser 

Encs. 
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Guidance on completing the questionnaire (appended to letter to 
survey respondents): factors affecting retention and/or achievement 



Ink or pencil? 

Please use black or blue ink to complete this questionnaire. 

Focus of the questionnaire 

Please complete the questionnaire with a focus on the course/programme identified at 
the beginning of the questionnaire. 

Period covered by questionnaire 

The questionnaire covers the period of the 3 academic years: 1997/1998, 1998/1999 
and 1999/2000. Please answer the questions in relation to this period. There is a 
separate section on the last page for you to identify changes with a significant impact in 
the current academic year. 

Factual section of questionnaire 

The questionnaire begins with a short factual section about this course/programme. 

Factors affecting retention and/or achievement 

This is the main section of the questionnaire and consists of lists of 13 sorts of factor 
from Curriculum (Section A) to Timetabling (Section M). 

Each factor has 2 boxes indicating the amount of change: 

Little Significant 

or no change 

change 

A B 

□ □ 

Please tick the relevant box to indicate whether there has been little or no change or 
significant change in the relevant factor. 
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Each factor also has five boxes indicating the impact of changes on student retention 
and/or achievement. Please tick the relevant box to indicate whether the change has 
had: 



a large negative impact (column 1 ) 
a moderate negative impact (column 2) 
no impact (column 3) 
a moderate positive impact (column 4) 
a large positive impact (5). 



Example 

If course/syllabus content (Factor A1 ) has changed a lot and the change has had a 
moderate positive impact on retention and/or achievement, your completed answer will 
look like this: 



Little 
or no 
change 

A 



Significant 

change 

B 



large no large 

negative impact positive 

impact impact 

1 2 3 4 5 




A1 Courses/syllabus content 




Free text box at end of each section 

At the end of each of the 13 sections there is a free text box. This is to allow you to give 
some examples of the sorts of changes that have had the largest impact (whether 
positive or negative) on student retention and/or achievement. Please use this box to 
describe changes, which you have scored 1 (large negative impact) or 5 (large positive 
impact). 

Changes with largest impact 

On the last page of the questionnaire there is a section that asks you to identify the 
changes which had the largest impact. Please tick the 3 factors where changes have 
had the largest negative impact and also the three factors where changes have had the 
largest positive impact on retention and/or achievement. 

2000/2001 academic year 

Please use the free text section at the end of the questionnaire to identify changes in 
the current academic year, which are likely to have a significant impact on retention 
and/or achievement for this course/programme. 




87 

87 



Appendix 4: Survey questionnaire 

CHANGES AFFECTING 
RETENTION & ACHIEVEMENT 

NAME OF COURSE OR PROGRAMME 



FACTUAL SECTION RELATING TO THE COURSE OR PROGRAMME 



learning 
and skills 
development 
agency 



What is the level of the course/programme? 

| | Level 1 □ Level 2 □ Level 3 



In 1999-2000, what type of qualification did the course/programme lead to? 

GNVQ 
Other 





GCSE 




AS or A level 






National Diploma 




NVQ 


_ 



If 'Other', please say what 



Is the course: 

□ Full Time I I Part Time 



How many hours of teaching per week does this course have? 



How long does the course last? 

I I One year I I Two years □ Other 



If 'Other', please state the length of course/programme 



What age are most students on the course? 

□ 16-19 □ 19 + 



How many staff teach on this course/programme? 

□ One □ 2 - 3 □ 4 - 5 D6-10 □ more than 10 

What role(s) do you carry out on the course / programme? 

(Please tick all boxes that apply.) 




I | Teacher 
I | Other 

(Please specify) 


I | T utor 


1 1 Course leader 










Survey : 535 
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in ii ill min 


III llll II 





Learnina & Skills DevelODment Aaencv 



FACTORS AFFECTING STUDENT RETENTION/ACHIEVEMENT 



In relation to this course / programme: 

1) How much change has there been to each of the factors over the three academic years 
1997 -2000? 

Please indicate by ticking one of the boxes to the left of each statement 

(A = little or no change or B = significant change) 

2) How have the changes affected retention /achievement over the three academic years 
1997 -2000? 

Please indicate this by ticking the relevant boxes: where: 1 = large negative impact and 5 = large 
positive impact. Please use the accompanying free text box to illustrate the sort of changes that have 
occurred if you have scored either 1 or 5. For ease of reference, the factors have been grouped into a 
number of categories. Changes in the current (2000 - 2001) academic year should be recorded 
separately at the end of the questionnaire. 



A: CURRICULUM 

How have changes to any of the curriculum factors impacted on retention/achievement over' the three 
academic years 1997 - 2000: 



little significant 
or no change 
change 



large no 

negative impact 
impact 



A B 12 3 

| | A1. Course/syllabus content I | f 

A2. Course/programme structure 

A3. Degree of difficulty of course/programme for students _ 

A4. Awarding body 

A5. Scheme of work 

1 I A6. Way in which scheme of work introduced to students _ 

| A7. Amount of enrichment linked to course/programme P] j 1 

~ _ A8. Amount of enrichment not linked to course/programme — 

A9. Tutoring arrangements — 

A10. Arrangements for Key Skills — 

LJ All. Any other curriculum factor (please specify which, below) — 



large 

positive 

impact 



A1 2. Overall, how have changes to curriculum factors 1 2 3 4 5 

impacted on retention and achievement? I I [ I fl FI f~l 

Please use the box below to describe the change for any factors that you have scored 1 or 5, using the 
reference number (e.g. A1) to indicate the factor you are referring to. 




Survey : 535 
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B: CONTEXT OF YOUR COURSE/PROGRAMME 



A B 

□ n B1 . Quality of internal communications in your team 
~ ~ B2. Clarity of what is expected of you by your line manager 

B3. How successes are dealt with by your course leader 

B4. How problems are dealt with by your course leader 

B5. Morale within your team 

B6. Morale within your department or faculty 

B7. Whether innovation in your teaching is encouraged 

B8. Emphasis placed on improving teaching and learning 

| | B9. Any other context factor (please specify which, below) 



1 2 3 4 5 




BIO. Overall, how have changes to the context of the programme 1 2 3 4 5 

factors impacted on retention and achievement? | | | | | | | | | | 



Please use the box below to describe the change for any factors that you have scored 1 or 5, using the 
reference number (e.g. B2) to indicate the factor you are referring to. 



C: ASSESSMENT AND FEEDBACK 



A B 

Cl. Students' understanding of assessment 

! C2. Amount of assessment 

C3. Methods of assessment 

C4. Scheduling of assessment 

C5. Policy on deadlines for handing work in 

C6. Amount of private study or homework 

C7. Helpfulness of feedback to students 

| | C8. Any other assessment/feedback factor 

(please specify which, below) 



1 2 3 4 5 




C9. Overall, how have changes to assessment and feedback 1 2 3 4 5 

factors impacted on retention and achievement? | | | | | | | | | | 

Please use the box below to describe the change for any factors that you have scored 1 or 5, using the 
reference number (e.g. C3) to indicate the factor you are referring to. 



Survey : 535 

hi ii hi min 



90 



Page : 3 

i 



Learning & Skills Development Agency 



90 



D: STAFFING 



4 5 



A B 

3 B 



D1 . Membership of your course/programme team 
D2. Mix of skills in your course/programme team 
D3. Your course/programme team leadership 
D4. Ratio of full to part time teachers in your course/ 
programme team 

D5. Amount of staff absence in your team 
D6. Your department/faculty headship 
D7. Senior curriculum leadership in your college 
D8. Composition of the group of personal tutors 
who tutor your students 

D9. Any other staffing factor (please specify which, below) 




DIO. Overall, how have changes to staffing factors impacted 
on retention and achievement? 



1 2 3 4 5 

□ □ □ □ □ 



Please use the box below to describe the change for any factors that you have scored 1 or 5, using the 
reference number (e g. D4) to indicate the factor you are referring to. 



E: QUALITY SYSTEMS 

1 2 3 4 5 

El. Self-assessment process for your course/programme 
E2. Availability of information from MIS 
E3. Observation of teaching and learning 
E4. Observation of tutoring 

E5. Processes for generating feedback from students 
E6. Use you make of feedback from students 
E7. Ways that managers monitor performance on your course/ 
programme 

E8. Any other quality systems factor 
(please specify which, below) 





E9. Overall, how have changes to quality system factors 
impacted on retention and achievement? 



1 2 3 4 5 

□ □ □ □ □ 



Please use the box below to describe the change for any factors that you have scored 1 or 5, using the 
reference number (e.g. E5) to indicate the factor you are referring to. 




Survey : 535 
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A B 



F: RESOURCES 



11 2 3 4 5 





FI . Time you have available for preparation 
F2. Number of teaching hours you have 
F3. Number of different courses you teach on 
F4. Availability of essential equipment for your course/ 
programme 

F5. Availability of technical support and backup 

F6. Accommodation for your course/programme 

F7. Any other resource factor (please specify which, below) 





F8. Overall, how have changes to resource factors 1 2 3 4 5 

impacted on retention and achievement? [ | [ | | | | | [ | 

Please use the box below to describe the change for any factors that you have scored 1 or 5, using the 
reference number (e.g. F6) to indicate the factor you are referring to. 



A 






G: STUDENT COHORT 

G1 . Motivation of students on your course/programme 
G2. Ability of students 

G3. Qualifications of students at commencement of the 
course/programme 
G4. Learning skills of students 
G5. Clarity of personal/career goals of students 
G6. Proportion of students leaving for a job 
G7. Amount of paid part time work done by students 
G8. How near to college your students live 
G9. Financial problems of students 
G10. Personal/emotional/health problems of students 
G1 1 . Any other student cohort factor 
(please specify which, below) 





4 




5 





G1 2. Overall, how have changes to student cohort factors 1 2 3 4 5 

impacted on retention and achievement? I II II II II I 

Please use the box below to describe the change for any factors that you have scored 1 or 5, using the 
reference number (e.g. G7) to indicate the factor you are referring to. 
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H: SUPPORT FOR STUDENTS 



A B 

BB 



HI. Involvement of parents of younger students 
H2. Ways that students are encouraged to support each other 
H3. Financial support available 
H4. Counselling and guidance support available 
H5. Availability of staff (personal tutor/subject teachers) 
to provide individual support when needed 
H6. Take up of learning support by students 
H7. Integration of learning support support within your course/ 
programme 

H8. Length of time it takes to identify "at risk" students 
H9. Amount of support available for "at risk" students 
H10. Methods of acknowledging/rewarding student progress 
H1 1 . Any other support for students factor 
(please specify which, below) 



1 2 3 4 5 




BBS 




B 





HI 2. Overall, how have changes to support for students factors 1 2 3 4 5 

impacted on retention and achievement? | [ | [ | | [ [ [ | 



Please use the box below to describe the change to any factors that you have scored 1 or 5, using the 
reference number (e.g. H8) to indicate the factor you are referring to. 



A B 

□ □ 



I: SUPPORT FOR TEACHERS 

11. Support available to you from within your team 

12. Support available to you from within our department 
or faculty 

13. Support available to you from within your college 

14. Mechanisms to acknowledge/reward effective teaching 

15. Opportunities available for staff development 

16. Administrative/clerical support 

17. Any other support for teachers factor 
(please specify which, below) 



1 2 3 4 5 

□ □ □ □ □ 




18. Overall, how have changes to support for teacher factors 1 2 3 4 5 

impacted on retention and achievement? | | fB I I f] I I 

Please use the box below to describe the change for any factors that you have scored 1 or 5, using the 
reference number (e.g. 17) to indicate the factor you are referring to 
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J: TEACHING 



B 



J1 . Your role as a teacher 
J2. Recruitment and selection processes 
J3. Entry requirements/admissions policy 
J4. Induction processes 

J5. Ways you monitor individual student progress 
J6. Variety of teaching strategies you use 
J7. Amount of teaching time involving different tasks 



J9. Proportion of teaching time involving "active learning" 
by students 

J10. Your involvement in development activity inside your college 
J1 1 . Your involvement in professional activity outside the college 
J12. Any other teaching factor (please specify which, below) 



□ 


□ 


for lower (or higher) attaining students 
J8. Amount of teaching time available to work with 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


individual students 


□ 


□ 


□ 


□ 


□ 



J1 3. Overall, how have changes to teaching factors impacted 1 2 3 4 5 

on retention and achievement? D D D D D 

Please use the box below to describe the change for any factors that you have scored 1 or 5, using the 
reference number (e g. J6) to indicate the factor you are referring to. 



K: PERSONAL TUTORING (ON THIS COURSE/PROGRAMME) 



A 



B 



1 2 3 4 5 



K1 . Role of the personal tutor 

K2. Amount of time students have for one-to-one tutorials 
K3. Amounts of time students have for group tutorials 
K4. Target setting processes in personal tutorials 
K5. Review and action planning processes in personal tutorials 
K6. Any other personal tutoring factor 
(please specify which, below) 




K7. Overall, how have changes to personal tutoring factors 1 2 3 4 5 

impacted on retention and achievement? I I | | | | I I I I 

Please use the box below to describe the change for any factors that you have scored 1 or 5, using the 
reference number (e.g. K5) to indicate the factor you are referring to. 
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L: COURSE/PROGRAMME ORGANISATION AND ADMINISTRATION 



A B 



□ □ 



LI. Ways you monitor and record student progress 
L2. Amount of course administration 
L3. Time available for team meetings 
L4. Effectiveness of team meetings 
L5. Team use of shared learning materials 
L6. Team involvement in planning all aspects of course 
L7. Amount of team teaching 
L8. Any other course/programme organisation and 
administration factor (please specify which, below) 



1 2 3 4 5 




□ □ □ □ □ 



L9. Overall, how have changes to course organisation and 1 2 3 4 5 

administration factors impacted on retention and achievement? Q I I | | | | | | 



Please use the box below to describe the change for any factors that you have scored 1 or 5, using the 
reference number (e.g. L4) to indicate the factor you are referring to. 



M: TIMETABLING 

A B 1 

Ml. Hours within which the course/programme 
1 n is taught each day — 

M2. Amount of gaps in timetable — 

M3. Number of weeks the course is taught over the year — 

M4. Amount of timetabled private study time for students — 

M5. Amount of time on timetable for students to "catch up" — 

I | M6. Any other timetabling factor (please specify which, below) ~ 



M7. Overall, how have changes to timtabling factors impacted 1 2 3 4 5 

on retention and achievement? | | [ | [ | | | [ | 



2 3 4 5 




Please use the box below to describe the change for any factors that you have scored 1 or 5, using the 
reference number (e.g. M3) to indicate the factor you are referring to. 
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Changes with largest impact 



Below is a list of 13 types of factors where you can tick those with the largest positive or negative impact. 
Please put a tick against the 3 factors where changes have had the largest negative impact and also the 3 
factors where changes have had the largest positive impact on retention/ achievement. 

Negative Positive 

□ A. CURRICULUM p 

_ B. CONTEXT OF YOUR COURSE — 

_ C. ASSESSMENT AND FEEDBACK 

_ D. STAFFING — 

□ E. QUALITY SYSTEMS p 

□ F. RESOURCES P 

_ G. STUDENT COHORT ~~ 

_ H. SUPPORT FOR STUDENTS p 

__ I. SUPPORT FOR TEACHERS ~ 

□ J. TEACHING p 

P| K. PERSONAL TUTORING □ 

□ L. COURSE ORGANISATION 

AND ADMINISTRATION p 

□ M. TIMETABLING p 



2000 - 2001 Academic Year 

Please use the box below to identify any changes in the current academic year which are likely to have a 
significant impact on retention/achievement. 



Thank you for taking the time to fill in this form. It is much appreciated. 

Please return the questionnaire to: 

Paul Martinez, Learning & Skills Development Agency, FREEPOST (BS 6745), London, SE11 5BF 

by: 

Friday 11th May 2001 
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Appendix 5: Guidance notes for college coordinators 



1 Selection of courses/programmes: introduction 

1.1 This exercise depends on being able to compare like with like so far as possible. 
We have agreed, therefore, to include courses/programmes within a ‘sampling 
frame’ and then select from that sample. 

2. Sampling frame 

2.1 Courses/programmes to be included in the sampling frame will have the 
following characteristics. 

♦ For 1-year courses, to be running in each of the 3 academic years 1997/8 - 
1999/2000 

♦ for courses lasting longer than 1 year, to have had a final year in each of the 
same 3 academic years 

♦ to have no less than 12 ‘starters’ (ie students who start and have not 
withdrawn before 1 November in the relevant academic year) 

♦ are at Level 1 , 2 or 3 

♦ are ‘long’ courses (ie with a minimum of 120 guided learning hours in a year) 

♦ the course/programme is or comprises part of a student’s primary learning 
goal. 

2.2 It is a matter of judgement whether changes to the course interrupt continuity. As 
a rule of thumb, the test is one of the degree of continuity. Thus, moving 
between GNVQ and National Diploma as a qualification aim would not 
necessarily interrupt continuity. 

2.3 Work- or employer-based programmes should be excluded from the sampling 
frame. This is because the employer variable is likely to be so significant that 
such programmes would require a separate research study. 

3 . Selection of sample of courses 

3.1 Please select 3 courses/programmes within each of the 4 boxes of the matrix set 
out below (ie a total of 12 courses). 



0 




97 



97 



1996/7 



Course D 


Course C 


Course B 


Course A 



Top 20% 

Baseline position at end of 
1996/7 academic year 

Bottom 20% 



Bottom 20% Top 20% 

Extent of improvement between 
September 1997 and July 2000 

3.2 Examples: 

Course A was in the bottom 20% of courses at the end of the 1996/7 
academic year, but in the 20% of courses, which improved most ,1997-2000. 

Course B was in the bottom 20% of courses at the end of the 1996/7 
academic year and in the 20% of courses which improved least. 

Course C was in the top 20% of courses at the end of the 1996/7 academic 
year and in the 20% of courses which improved most. 

Course D was in the top 20% of courses at the end of the 1996/7 academic 
year, but in the 20% of courses which improved least. 

3.3 The baseline position and extent of improvement should be calculated in relation 
to your own college’s norms. In other words, you will select programmes in the 
top or bottom 20% by reference to your own college’s data rather than national 
norms. 

3.4 The criterion in respect of the calculation of both the baseline position and the 
rate of improvement is the ‘success ratio’. The success ratio represents the 
percentage of achievers over those students who have started the course (and 
who did not withdraw prior to 1 November). 

3.5 Simply put, the success ratio is the product of multiplying the percentage 
retention rate by the percentage achievement rate. 

3.6 In respect of the improvement criteria, this should be calculated cumulatively 
from September 1997 to July 2000. 

3.7 In respect of the baseline criteria this should be calculated by reference to the 
position at the end of the 1996/7 academic year. 
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3.8 Fluctuation in a success ratio does not matter; it is the cumulative change over 
the 3-year period that counts. 

4. Selection of staff 

4.1 Whatever other role they may have (eg tutor, course team leader, etc) 
respondents should have a teaching commitment to the course/programme 
for at least 2 of the 3 years in question. For the purposes of the research, this 
will make them ‘qualifying staff. 

4.2 For courses/programmes with 5 or fewer ‘qualifying staff, give all ‘qualifying 
staff members of the team a copy of the questionnaire. 

4.3 For courses/programmes with 6 or more ‘qualifying staff, please distribute the 
questionnaire to a maximum of 5 ‘qualifying staff, including a mix of full-time and 
part-time staff as appropriate. 

5. Timing 

5.1 Please return the completed questionnaires to Graham Knight (at the Vauxhall, 
London office of the Agency) in the Freepost envelope provided to arrive on or 
before Friday 1 1 May. 

5.2 Please remember to include the data sheets (see section 7 below). 

5.3 We will send you sufficient copies of the questionnaire and letters to staff (60 per 
college), to arrive no later than the week commencing 9 April. 

6. Distribution of questionnaires 

6.1 Please distribute the questionnaires directly to the ‘qualifying’ members of staff 
who will be completing them. 

6.2 Staff may teach on a number of different programmes. To make the focus of the 
questionnaire clear, please write the title of the relevant course/programme in 
the beginning of the questionnaire and make sure this tallies with the title on the 
relevant data sheet (see section 7 below). 

6.3 Please hold a briefing session to deal with any questions and allay any 
anxieties. 

6.4 Please emphasise the importance of the exercise, its confidentiality and the 
need for more or less instantaneous completion of the questionnaire. 
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6.5 Please emphasise that the questionnaire is to be completed individually rather 
than as a team effort. 

6.6 It is worth reminding staff that this is an attempt to get at reality through a 
survey. Respondents should be encouraged to complete it fairly quickly without 
reference to course files, etc. 

6.7 Without being unduly prescriptive, you might find it easiest to invite people to a 
briefing, supply them liberally with coffee and biscuits and get them to complete 
the questionnaire on the spot. 

7. Course/programme data sheet 

7.1 You will receive a supply of course/programme data sheets separate from the 
questionnaires. 

7.2 Please ensure that a data sheet is completed for each of the 12 
courses/programmes you select to include in this survey. 

7.3 Please complete the data sheet from your central MIS records and do not 
distribute it in with the survey. 

7.4 Please return the completed data sheets along with the completed survey forms 
to Graham Knight in the envelope provided. It would be helpful if you would 
attach the set of completed questionnaires to the relevant course/programme 
data sheet. 

7.5 Please record the number of questionnaires distributed and the number returned 
for each programme/course on the data sheet. 

7.6 Finally, please indicate the type of the course in terms of its baseline and 
improvement. 
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Appendix 6: Data sheet for each course/prog ram me 



FOR CO-ORDINATOR’S USE ONLY: COURSE / PROGRAMME DATA 



Title of Course/ 
Programme 



For each of the last three academic years, please give the information requested in the table 
below. Please have regard to the Explanatory Notes given on the reverse of this page. 



COURSE / PROGRAMME DATA 



Column 1 Column 2 Column 3 Column 4 Column 5 



Academic Starts Retention Achievement Success 

Year (Number) (%) Rate (%) Rate (%) 



1999-2000 

1998-1999 

1997-1998 




Number of questionnaires 

How many questionnaires did you give out for this course? 
How many completed questionnaires did you get back? 



Type of course / programme 

Please tick the box which best describes this course: 



High baseline position, high improvement EH 

High baseline position, little or no improvement I I 

Low baseline position, high improvement I I 

Low baseline position, little or no improvement EH 
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Explanatory Notes : Course / Programme Data 



Column 1 - 


The academic year relates to the year the course completed. Thus, two-year 
courses completed in 1997/98 would need to show students enrolled as at 
1st November 1996. 


Column 2 - 


Please state the number of students who start and do not leave before 1st 
November in the same academic year for one-year courses and in the 
preceding year for two-year courses. 


Column 3 - 


Please calculate this percentage as follows: 

Students still enrolled on the 1st May expressed as a percentage of student 
starters who have not left before the previous 1st November for one-year 
courses and 1st November for the preceding year for two-year courses. 


Column 4 - 


Please calculate this percentage as follows: 

Students who achieve a pass in the qualification aim expressed as a 
percentage of completers. 


Column 5 - 


Please calculate this percentage as follows: 

Students who achieved a pass expressed as a percentage of starters in 
column 3 multiplied by the percentage of achievers in column 4. 



Number of questionnaires 

This allows us to identify the extent of any non-returns. Ideally we anticipate a 100% rate of 
return. 

Type of course / programme 

'Baseline' and 'improvement' are defined and discussed in Section 3 of the Guidance Notes for 
College Co-ordinators. 

BEST COPY AVAILABLE 
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raising quality 
and achievement 
programme 



© 




The Raising Quality and Achievement Programme 
is run by the Learning and Skills Development Agency 
in partnership with the Association of Colleges and 
the Centre for Economic and Social Inclusion. 

■ We aim to reach all colleges and work-based learning providers. 

■ We offer extra support to colleges and work-based 
learning providers that are receiving Standards Fund money 
to improve their practice. 

■ All our activity themes are backed by a programme 
of research and evaluation. 

■ The Raising Quality and Achievement Programme is funded 
by a grant to the Learning and Skills Development Agency 
from the Learning and Skills Council. 
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